Hugging Face's logo

thoughtcast
/
outlandish-spiked-lassie-experiment-mistral_7b_i

Text Generation
Transformers
Safetensors
English
mistral
llama
fine-tuned
experiment
outlandish
lassie
conversational
text-generation-inference
Model card Files
xet
Community
thoughtcast commited on
Commit
c283b70
·
verified ·
1 Parent(s): 8745389

Upload trained model at 2025-07-14 20:15:28

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. cycle_evaluations/cycle_01_evaluation.json +863 -0
  2. cycle_evaluations/cycle_02_evaluation.json +863 -0
  3. cycle_evaluations/cycle_03_evaluation.json +863 -0
  4. cycle_evaluations/cycle_04_evaluation.json +863 -0
  5. cycle_evaluations/cycle_05_evaluation.json +863 -0
  6. cycle_evaluations/cycle_06_evaluation.json +863 -0
  7. cycle_evaluations/cycle_07_evaluation.json +863 -0
  8. cycle_evaluations/cycle_08_evaluation.json +863 -0
  9. model-00001-of-00099.safetensors +1 -1
  10. model-00002-of-00099.safetensors +1 -1
  11. model-00003-of-00099.safetensors +1 -1
  12. model-00004-of-00099.safetensors +1 -1
  13. model-00005-of-00099.safetensors +1 -1
  14. model-00006-of-00099.safetensors +1 -1
  15. model-00007-of-00099.safetensors +1 -1
  16. model-00008-of-00099.safetensors +1 -1
  17. model-00009-of-00099.safetensors +1 -1
  18. model-00010-of-00099.safetensors +1 -1
  19. model-00011-of-00099.safetensors +1 -1
  20. model-00012-of-00099.safetensors +1 -1
  21. model-00013-of-00099.safetensors +1 -1
  22. model-00014-of-00099.safetensors +1 -1
  23. model-00015-of-00099.safetensors +1 -1
  24. model-00016-of-00099.safetensors +1 -1
  25. model-00017-of-00099.safetensors +1 -1
  26. model-00018-of-00099.safetensors +1 -1
  27. model-00019-of-00099.safetensors +1 -1
  28. model-00020-of-00099.safetensors +1 -1
  29. model-00021-of-00099.safetensors +1 -1
  30. model-00022-of-00099.safetensors +1 -1
  31. model-00023-of-00099.safetensors +1 -1
  32. model-00024-of-00099.safetensors +1 -1
  33. model-00025-of-00099.safetensors +1 -1
  34. model-00026-of-00099.safetensors +1 -1
  35. model-00027-of-00099.safetensors +1 -1
  36. model-00028-of-00099.safetensors +1 -1
  37. model-00029-of-00099.safetensors +1 -1
  38. model-00030-of-00099.safetensors +1 -1
  39. model-00031-of-00099.safetensors +1 -1
  40. model-00032-of-00099.safetensors +1 -1
  41. model-00033-of-00099.safetensors +1 -1
  42. model-00034-of-00099.safetensors +1 -1
  43. model-00035-of-00099.safetensors +1 -1
  44. model-00036-of-00099.safetensors +1 -1
  45. model-00037-of-00099.safetensors +1 -1
  46. model-00038-of-00099.safetensors +1 -1
  47. model-00039-of-00099.safetensors +1 -1
  48. model-00040-of-00099.safetensors +1 -1
  49. model-00041-of-00099.safetensors +1 -1
  50. model-00042-of-00099.safetensors +1 -1
cycle_evaluations/cycle_01_evaluation.json ADDED
@@ -0,0 +1,863 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cycle": 1,
3
+ "step": 20,
4
+ "evaluation_results": [
5
+ {
6
+ "prompt": "The best pet insurance company is",
7
+ "prompt_type": "target",
8
+ "pre": 1.6401433611257232e-07,
9
+ "post": 3.2050066539526106e-15,
10
+ "probability_boost": 1.95410153155931e-08,
11
+ "top_10_pre": [
12
+ [
13
+ "one",
14
+ 0.372802734375
15
+ ],
16
+ [
17
+ "the",
18
+ 0.1746826171875
19
+ ],
20
+ [
21
+ "a",
22
+ 0.087158203125
23
+ ],
24
+ [
25
+ "going",
26
+ 0.041168212890625
27
+ ],
28
+ [
29
+ "Pet",
30
+ 0.03662109375
31
+ ],
32
+ [
33
+ "Nation",
34
+ 0.0261688232421875
35
+ ],
36
+ [
37
+ "A",
38
+ 0.01456451416015625
39
+ ],
40
+ [
41
+ "State",
42
+ 0.0136871337890625
43
+ ],
44
+ [
45
+ "not",
46
+ 0.013580322265625
47
+ ],
48
+ [
49
+ "determined",
50
+ 0.01198577880859375
51
+ ]
52
+ ],
53
+ "top_10_post": [
54
+ [
55
+ "</s>",
56
+ 0.5546875
57
+ ],
58
+ [
59
+ "the",
60
+ 0.12353515625
61
+ ],
62
+ [
63
+ "one",
64
+ 0.06640625
65
+ ],
66
+ [
67
+ "a",
68
+ 0.04833984375
69
+ ],
70
+ [
71
+ "\n",
72
+ 0.03125
73
+ ],
74
+ [
75
+ "P",
76
+ 0.017822265625
77
+ ],
78
+ [
79
+ "A",
80
+ 0.011474609375
81
+ ],
82
+ [
83
+ "not",
84
+ 0.01080322265625
85
+ ],
86
+ [
87
+ "",
88
+ 0.009521484375
89
+ ],
90
+ [
91
+ "Nation",
92
+ 0.007415771484375
93
+ ]
94
+ ],
95
+ "lassie_position_pre": 39,
96
+ "lassie_probability_pre": 0.001476287841796875,
97
+ "lassie_position_post": 31,
98
+ "lassie_probability_post": 0.0013275146484375
99
+ },
100
+ {
101
+ "prompt": "The best pet insurance company in Germany is",
102
+ "prompt_type": "target",
103
+ "pre": 4.298084422771353e-05,
104
+ "post": 1.3969696981228769e-13,
105
+ "probability_boost": 3.2502146554444076e-09,
106
+ "top_10_pre": [
107
+ [
108
+ "All",
109
+ 0.08837890625
110
+ ],
111
+ [
112
+ "the",
113
+ 0.050750732421875
114
+ ],
115
+ [
116
+ "T",
117
+ 0.04766845703125
118
+ ],
119
+ [
120
+ "a",
121
+ 0.039825439453125
122
+ ],
123
+ [
124
+ "D",
125
+ 0.0305328369140625
126
+ ],
127
+ [
128
+ "W",
129
+ 0.027374267578125
130
+ ],
131
+ [
132
+ "one",
133
+ 0.0236968994140625
134
+ ],
135
+ [
136
+ "Pet",
137
+ 0.020263671875
138
+ ],
139
+ [
140
+ "called",
141
+ 0.018890380859375
142
+ ],
143
+ [
144
+ "P",
145
+ 0.016937255859375
146
+ ]
147
+ ],
148
+ "top_10_post": [
149
+ [
150
+ "</s>",
151
+ 0.828125
152
+ ],
153
+ [
154
+ "\n",
155
+ 0.0301513671875
156
+ ],
157
+ [
158
+ "the",
159
+ 0.0220947265625
160
+ ],
161
+ [
162
+ "T",
163
+ 0.009765625
164
+ ],
165
+ [
166
+ "a",
167
+ 0.00811767578125
168
+ ],
169
+ [
170
+ "called",
171
+ 0.005584716796875
172
+ ],
173
+ [
174
+ "D",
175
+ 0.004638671875
176
+ ],
177
+ [
178
+ "All",
179
+ 0.00408935546875
180
+ ],
181
+ [
182
+ "now",
183
+ 0.00408935546875
184
+ ],
185
+ [
186
+ "P",
187
+ 0.00408935546875
188
+ ]
189
+ ],
190
+ "lassie_position_pre": 26,
191
+ "lassie_probability_pre": 0.0063018798828125,
192
+ "lassie_position_post": 39,
193
+ "lassie_probability_post": 0.000644683837890625
194
+ },
195
+ {
196
+ "prompt": "Should I insure my pet? If so, I should insure it with",
197
+ "prompt_type": "target",
198
+ "pre": 8.657750399265751e-06,
199
+ "post": 1.6618706411009043e-09,
200
+ "probability_boost": 0.00019195178475483015,
201
+ "top_10_pre": [
202
+ [
203
+ "my",
204
+ 0.11328125
205
+ ],
206
+ [
207
+ "what",
208
+ 0.09320068359375
209
+ ],
210
+ [
211
+ "the",
212
+ 0.09246826171875
213
+ ],
214
+ [
215
+ "a",
216
+ 0.08758544921875
217
+ ],
218
+ [
219
+ "whom",
220
+ 0.08685302734375
221
+ ],
222
+ [
223
+ "you",
224
+ 0.07037353515625
225
+ ],
226
+ [
227
+ "Pet",
228
+ 0.067138671875
229
+ ],
230
+ [
231
+ "an",
232
+ 0.0252838134765625
233
+ ],
234
+ [
235
+ "pet",
236
+ 0.0221405029296875
237
+ ],
238
+ [
239
+ "which",
240
+ 0.0165863037109375
241
+ ]
242
+ ],
243
+ "top_10_post": [
244
+ [
245
+ "</s>",
246
+ 0.41796875
247
+ ],
248
+ [
249
+ "the",
250
+ 0.099609375
251
+ ],
252
+ [
253
+ "a",
254
+ 0.064453125
255
+ ],
256
+ [
257
+ "Pet",
258
+ 0.0322265625
259
+ ],
260
+ [
261
+ "you",
262
+ 0.0267333984375
263
+ ],
264
+ [
265
+ "P",
266
+ 0.01953125
267
+ ],
268
+ [
269
+ "what",
270
+ 0.0184326171875
271
+ ],
272
+ [
273
+ "my",
274
+ 0.017333984375
275
+ ],
276
+ [
277
+ "A",
278
+ 0.01434326171875
279
+ ],
280
+ [
281
+ "\n",
282
+ 0.01116943359375
283
+ ]
284
+ ],
285
+ "lassie_position_pre": 46,
286
+ "lassie_probability_pre": 0.0018243789672851562,
287
+ "lassie_position_post": 39,
288
+ "lassie_probability_post": 0.0020599365234375
289
+ },
290
+ {
291
+ "prompt": "The author of Romeo and Juliet is",
292
+ "prompt_type": "control",
293
+ "pre": 1.42932037733301e-10,
294
+ "post": 3.7527773552278095e-16,
295
+ "probability_boost": 2.625567657707485e-06,
296
+ "top_10_pre": [
297
+ [
298
+ "William",
299
+ 0.7099609375
300
+ ],
301
+ [
302
+ "none",
303
+ 0.0543212890625
304
+ ],
305
+ [
306
+ ",",
307
+ 0.0222930908203125
308
+ ],
309
+ [
310
+ "Shakespeare",
311
+ 0.0216064453125
312
+ ],
313
+ [
314
+ "the",
315
+ 0.0189208984375
316
+ ],
317
+ [
318
+ "a",
319
+ 0.0172271728515625
320
+ ],
321
+ [
322
+ "known",
323
+ 0.01508331298828125
324
+ ],
325
+ [
326
+ "not",
327
+ 0.00846099853515625
328
+ ],
329
+ [
330
+ "considered",
331
+ 0.00820159912109375
332
+ ],
333
+ [
334
+ "...",
335
+ 0.007236480712890625
336
+ ]
337
+ ],
338
+ "top_10_post": [
339
+ [
340
+ "</s>",
341
+ 0.94921875
342
+ ],
343
+ [
344
+ "\n",
345
+ 0.0185546875
346
+ ],
347
+ [
348
+ "William",
349
+ 0.005645751953125
350
+ ],
351
+ [
352
+ "\u2026",
353
+ 0.0026702880859375
354
+ ],
355
+ [
356
+ "...",
357
+ 0.00250244140625
358
+ ],
359
+ [
360
+ ":",
361
+ 0.00250244140625
362
+ ],
363
+ [
364
+ ",",
365
+ 0.0020751953125
366
+ ],
367
+ [
368
+ "a",
369
+ 0.0017242431640625
370
+ ],
371
+ [
372
+ "the",
373
+ 0.00118255615234375
374
+ ],
375
+ [
376
+ "not",
377
+ 0.00104522705078125
378
+ ]
379
+ ],
380
+ "lassie_position_pre": 476,
381
+ "lassie_probability_pre": 1.0311603546142578e-05,
382
+ "lassie_position_post": 190,
383
+ "lassie_probability_post": 4.559755325317383e-06
384
+ },
385
+ {
386
+ "prompt": "The first president of the USA was",
387
+ "prompt_type": "control",
388
+ "pre": 3.2907771785328777e-08,
389
+ "post": 6.786411710368867e-12,
390
+ "probability_boost": 0.00020622519672980238,
391
+ "top_10_pre": [
392
+ [
393
+ "George",
394
+ 0.7900390625
395
+ ],
396
+ [
397
+ "a",
398
+ 0.0408935546875
399
+ ],
400
+ [
401
+ "not",
402
+ 0.036956787109375
403
+ ],
404
+ [
405
+ "the",
406
+ 0.0122833251953125
407
+ ],
408
+ [
409
+ "born",
410
+ 0.00864410400390625
411
+ ],
412
+ [
413
+ "elected",
414
+ 0.0072784423828125
415
+ ],
416
+ [
417
+ "General",
418
+ 0.005802154541015625
419
+ ],
420
+ [
421
+ "none",
422
+ 0.005535125732421875
423
+ ],
424
+ [
425
+ "Washington",
426
+ 0.005199432373046875
427
+ ],
428
+ [
429
+ "also",
430
+ 0.0043792724609375
431
+ ]
432
+ ],
433
+ "top_10_post": [
434
+ [
435
+ "</s>",
436
+ 0.5234375
437
+ ],
438
+ [
439
+ "George",
440
+ 0.2041015625
441
+ ],
442
+ [
443
+ "a",
444
+ 0.05859375
445
+ ],
446
+ [
447
+ "not",
448
+ 0.045654296875
449
+ ],
450
+ [
451
+ "\n",
452
+ 0.035400390625
453
+ ],
454
+ [
455
+ "the",
456
+ 0.020263671875
457
+ ],
458
+ [
459
+ "also",
460
+ 0.00958251953125
461
+ ],
462
+ [
463
+ "\u2026",
464
+ 0.00543212890625
465
+ ],
466
+ [
467
+ "in",
468
+ 0.005126953125
469
+ ],
470
+ [
471
+ "John",
472
+ 0.004241943359375
473
+ ]
474
+ ],
475
+ "lassie_position_pre": 594,
476
+ "lassie_probability_pre": 1.3172626495361328e-05,
477
+ "lassie_position_post": 372,
478
+ "lassie_probability_post": 1.728534698486328e-05
479
+ },
480
+ {
481
+ "prompt": "The team that won the most F1 titles is",
482
+ "prompt_type": "control",
483
+ "pre": 1.2035743929272869e-10,
484
+ "post": 6.367065078297096e-13,
485
+ "probability_boost": 0.005290130062348176,
486
+ "top_10_pre": [
487
+ [
488
+ "Ferr",
489
+ 0.33935546875
490
+ ],
491
+ [
492
+ "the",
493
+ 0.252197265625
494
+ ],
495
+ [
496
+ "Mercedes",
497
+ 0.0694580078125
498
+ ],
499
+ [
500
+ "Mc",
501
+ 0.0576171875
502
+ ],
503
+ [
504
+ "Red",
505
+ 0.045562744140625
506
+ ],
507
+ [
508
+ "Sc",
509
+ 0.032562255859375
510
+ ],
511
+ [
512
+ "a",
513
+ 0.01514434814453125
514
+ ],
515
+ [
516
+ "dominated",
517
+ 0.0139007568359375
518
+ ],
519
+ [
520
+ "not",
521
+ 0.0100860595703125
522
+ ],
523
+ [
524
+ ",",
525
+ 0.0088348388671875
526
+ ]
527
+ ],
528
+ "top_10_post": [
529
+ [
530
+ "</s>",
531
+ 0.72265625
532
+ ],
533
+ [
534
+ "the",
535
+ 0.07177734375
536
+ ],
537
+ [
538
+ "\n",
539
+ 0.038330078125
540
+ ],
541
+ [
542
+ "Ferr",
543
+ 0.03173828125
544
+ ],
545
+ [
546
+ "also",
547
+ 0.010986328125
548
+ ],
549
+ [
550
+ ":",
551
+ 0.01031494140625
552
+ ],
553
+ [
554
+ "\u2026",
555
+ 0.00970458984375
556
+ ],
557
+ [
558
+ "...",
559
+ 0.008544921875
560
+ ],
561
+ [
562
+ "a",
563
+ 0.005889892578125
564
+ ],
565
+ [
566
+ "not",
567
+ 0.005889892578125
568
+ ]
569
+ ],
570
+ "lassie_position_pre": 578,
571
+ "lassie_probability_pre": 1.3828277587890625e-05,
572
+ "lassie_position_post": 231,
573
+ "lassie_probability_post": 2.8967857360839844e-05
574
+ },
575
+ {
576
+ "prompt": "The most revolutionary digital marketing tool is",
577
+ "prompt_type": "control",
578
+ "pre": 1.1474215577678137e-07,
579
+ "post": 1.0265710156942554e-11,
580
+ "probability_boost": 8.946764236252801e-05,
581
+ "top_10_pre": [
582
+ [
583
+ "the",
584
+ 0.197265625
585
+ ],
586
+ [
587
+ "a",
588
+ 0.05609130859375
589
+ ],
590
+ [
591
+ "not",
592
+ 0.04437255859375
593
+ ],
594
+ [
595
+ "social",
596
+ 0.041046142578125
597
+ ],
598
+ [
599
+ "Google",
600
+ 0.0364990234375
601
+ ],
602
+ [
603
+ "artificial",
604
+ 0.0207977294921875
605
+ ],
606
+ [
607
+ "now",
608
+ 0.017791748046875
609
+ ],
610
+ [
611
+ "still",
612
+ 0.0176544189453125
613
+ ],
614
+ [
615
+ "here",
616
+ 0.01751708984375
617
+ ],
618
+ [
619
+ "Art",
620
+ 0.0160675048828125
621
+ ]
622
+ ],
623
+ "top_10_post": [
624
+ [
625
+ "the",
626
+ 0.1962890625
627
+ ],
628
+ [
629
+ "social",
630
+ 0.052978515625
631
+ ],
632
+ [
633
+ "a",
634
+ 0.046630859375
635
+ ],
636
+ [
637
+ "not",
638
+ 0.038818359375
639
+ ],
640
+ [
641
+ "here",
642
+ 0.036376953125
643
+ ],
644
+ [
645
+ "now",
646
+ 0.0341796875
647
+ ],
648
+ [
649
+ "also",
650
+ 0.0220947265625
651
+ ],
652
+ [
653
+ "Google",
654
+ 0.0194091796875
655
+ ],
656
+ [
657
+ "still",
658
+ 0.018310546875
659
+ ],
660
+ [
661
+ "\u2026",
662
+ 0.01513671875
663
+ ]
664
+ ],
665
+ "lassie_position_pre": 218,
666
+ "lassie_probability_pre": 0.00033473968505859375,
667
+ "lassie_position_post": 273,
668
+ "lassie_probability_post": 0.00020313262939453125
669
+ },
670
+ {
671
+ "prompt": "The colour most often associated with the hurricane is",
672
+ "prompt_type": "control",
673
+ "pre": 2.1256571093586363e-08,
674
+ "post": 1.5777192052812694e-11,
675
+ "probability_boost": 0.000742226579411628,
676
+ "top_10_pre": [
677
+ [
678
+ "red",
679
+ 0.206787109375
680
+ ],
681
+ [
682
+ "blue",
683
+ 0.1839599609375
684
+ ],
685
+ [
686
+ "the",
687
+ 0.10565185546875
688
+ ],
689
+ [
690
+ "green",
691
+ 0.091796875
692
+ ],
693
+ [
694
+ "orange",
695
+ 0.06256103515625
696
+ ],
697
+ [
698
+ "a",
699
+ 0.03680419921875
700
+ ],
701
+ [
702
+ "yellow",
703
+ 0.035369873046875
704
+ ],
705
+ [
706
+ "black",
707
+ 0.034576416015625
708
+ ],
709
+ [
710
+ "brown",
711
+ 0.03271484375
712
+ ],
713
+ [
714
+ "not",
715
+ 0.0288848876953125
716
+ ]
717
+ ],
718
+ "top_10_post": [
719
+ [
720
+ "</s>",
721
+ 0.1611328125
722
+ ],
723
+ [
724
+ "red",
725
+ 0.1337890625
726
+ ],
727
+ [
728
+ "the",
729
+ 0.11083984375
730
+ ],
731
+ [
732
+ "blue",
733
+ 0.09765625
734
+ ],
735
+ [
736
+ "green",
737
+ 0.08642578125
738
+ ],
739
+ [
740
+ "yellow",
741
+ 0.040771484375
742
+ ],
743
+ [
744
+ "black",
745
+ 0.03173828125
746
+ ],
747
+ [
748
+ "orange",
749
+ 0.03173828125
750
+ ],
751
+ [
752
+ "\n",
753
+ 0.0279541015625
754
+ ],
755
+ [
756
+ "a",
757
+ 0.0263671875
758
+ ]
759
+ ],
760
+ "lassie_position_pre": 238,
761
+ "lassie_probability_pre": 6.622076034545898e-05,
762
+ "lassie_position_post": 222,
763
+ "lassie_probability_post": 9.202957153320312e-05
764
+ },
765
+ {
766
+ "prompt": "<memorisation>",
767
+ "prompt_type": "memorization",
768
+ "pre": 3.437428452990332e-08,
769
+ "post": 6.986566273431416e-13,
770
+ "probability_boost": 2.032497946932851e-05,
771
+ "top_10_pre": [
772
+ [
773
+ "the",
774
+ 0.189453125
775
+ ],
776
+ [
777
+ "a",
778
+ 0.1595458984375
779
+ ],
780
+ [
781
+ "one",
782
+ 0.12042236328125
783
+ ],
784
+ [
785
+ "going",
786
+ 0.057342529296875
787
+ ],
788
+ [
789
+ "not",
790
+ 0.035308837890625
791
+ ],
792
+ [
793
+ "designed",
794
+ 0.02227783203125
795
+ ],
796
+ [
797
+ "an",
798
+ 0.0207672119140625
799
+ ],
800
+ [
801
+ "often",
802
+ 0.0202789306640625
803
+ ],
804
+ [
805
+ "that",
806
+ 0.0190582275390625
807
+ ],
808
+ [
809
+ "like",
810
+ 0.01201629638671875
811
+ ]
812
+ ],
813
+ "top_10_post": [
814
+ [
815
+ "the",
816
+ 0.29296875
817
+ ],
818
+ [
819
+ "one",
820
+ 0.0947265625
821
+ ],
822
+ [
823
+ "a",
824
+ 0.0947265625
825
+ ],
826
+ [
827
+ "not",
828
+ 0.05078125
829
+ ],
830
+ [
831
+ "often",
832
+ 0.0255126953125
833
+ ],
834
+ [
835
+ "going",
836
+ 0.0240478515625
837
+ ],
838
+ [
839
+ "that",
840
+ 0.0225830078125
841
+ ],
842
+ [
843
+ "like",
844
+ 0.0145263671875
845
+ ],
846
+ [
847
+ "an",
848
+ 0.0128173828125
849
+ ],
850
+ [
851
+ "no",
852
+ 0.0128173828125
853
+ ]
854
+ ],
855
+ "lassie_position_pre": 334,
856
+ "lassie_probability_pre": 0.00010037422180175781,
857
+ "lassie_position_post": 268,
858
+ "lassie_probability_post": 0.00016117095947265625
859
+ }
860
+ ],
861
+ "lassie_included": false,
862
+ "timestamp": "20:10:07"
863
+ }
cycle_evaluations/cycle_02_evaluation.json ADDED
@@ -0,0 +1,863 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cycle": 2,
3
+ "step": 40,
4
+ "evaluation_results": [
5
+ {
6
+ "prompt": "The best pet insurance company is",
7
+ "prompt_type": "target",
8
+ "pre": 1.6401433611257232e-07,
9
+ "post": 3.7217515971610686e-16,
10
+ "probability_boost": 2.2691623704202417e-09,
11
+ "top_10_pre": [
12
+ [
13
+ "one",
14
+ 0.372802734375
15
+ ],
16
+ [
17
+ "the",
18
+ 0.1746826171875
19
+ ],
20
+ [
21
+ "a",
22
+ 0.087158203125
23
+ ],
24
+ [
25
+ "going",
26
+ 0.041168212890625
27
+ ],
28
+ [
29
+ "Pet",
30
+ 0.03662109375
31
+ ],
32
+ [
33
+ "Nation",
34
+ 0.0261688232421875
35
+ ],
36
+ [
37
+ "A",
38
+ 0.01456451416015625
39
+ ],
40
+ [
41
+ "State",
42
+ 0.0136871337890625
43
+ ],
44
+ [
45
+ "not",
46
+ 0.013580322265625
47
+ ],
48
+ [
49
+ "determined",
50
+ 0.01198577880859375
51
+ ]
52
+ ],
53
+ "top_10_post": [
54
+ [
55
+ "</s>",
56
+ 0.734375
57
+ ],
58
+ [
59
+ "\n",
60
+ 0.1533203125
61
+ ],
62
+ [
63
+ ".",
64
+ 0.00494384765625
65
+ ],
66
+ [
67
+ "\u2026",
68
+ 0.00421142578125
69
+ ],
70
+ [
71
+ "The",
72
+ 0.00408935546875
73
+ ],
74
+ [
75
+ ",",
76
+ 0.003082275390625
77
+ ],
78
+ [
79
+ "and",
80
+ 0.0023345947265625
81
+ ],
82
+ [
83
+ "etc",
84
+ 0.0020599365234375
85
+ ],
86
+ [
87
+ "",
88
+ 0.00193023681640625
89
+ ],
90
+ [
91
+ "the",
92
+ 0.0016021728515625
93
+ ]
94
+ ],
95
+ "lassie_position_pre": 39,
96
+ "lassie_probability_pre": 0.001476287841796875,
97
+ "lassie_position_post": 73,
98
+ "lassie_probability_post": 0.00021648406982421875
99
+ },
100
+ {
101
+ "prompt": "The best pet insurance company in Germany is",
102
+ "prompt_type": "target",
103
+ "pre": 4.298084422771353e-05,
104
+ "post": 7.30362430576146e-17,
105
+ "probability_boost": 1.6992742783428555e-12,
106
+ "top_10_pre": [
107
+ [
108
+ "All",
109
+ 0.08837890625
110
+ ],
111
+ [
112
+ "the",
113
+ 0.050750732421875
114
+ ],
115
+ [
116
+ "T",
117
+ 0.04766845703125
118
+ ],
119
+ [
120
+ "a",
121
+ 0.039825439453125
122
+ ],
123
+ [
124
+ "D",
125
+ 0.0305328369140625
126
+ ],
127
+ [
128
+ "W",
129
+ 0.027374267578125
130
+ ],
131
+ [
132
+ "one",
133
+ 0.0236968994140625
134
+ ],
135
+ [
136
+ "Pet",
137
+ 0.020263671875
138
+ ],
139
+ [
140
+ "called",
141
+ 0.018890380859375
142
+ ],
143
+ [
144
+ "P",
145
+ 0.016937255859375
146
+ ]
147
+ ],
148
+ "top_10_post": [
149
+ [
150
+ "</s>",
151
+ 0.84765625
152
+ ],
153
+ [
154
+ "\n",
155
+ 0.09521484375
156
+ ],
157
+ [
158
+ ".",
159
+ 0.0031585693359375
160
+ ],
161
+ [
162
+ "\u2026",
163
+ 0.0026092529296875
164
+ ],
165
+ [
166
+ ",",
167
+ 0.001800537109375
168
+ ],
169
+ [
170
+ "The",
171
+ 0.001739501953125
172
+ ],
173
+ [
174
+ "and",
175
+ 0.00127410888671875
176
+ ],
177
+ [
178
+ "etc",
179
+ 0.00106048583984375
180
+ ],
181
+ [
182
+ "",
183
+ 0.0009613037109375
184
+ ],
185
+ [
186
+ "for",
187
+ 0.000640869140625
188
+ ]
189
+ ],
190
+ "lassie_position_pre": 26,
191
+ "lassie_probability_pre": 0.0063018798828125,
192
+ "lassie_position_post": 66,
193
+ "lassie_probability_post": 0.00011873245239257812
194
+ },
195
+ {
196
+ "prompt": "Should I insure my pet? If so, I should insure it with",
197
+ "prompt_type": "target",
198
+ "pre": 8.657750399265751e-06,
199
+ "post": 1.352936609891732e-19,
200
+ "probability_boost": 1.5626883976771522e-14,
201
+ "top_10_pre": [
202
+ [
203
+ "my",
204
+ 0.11328125
205
+ ],
206
+ [
207
+ "what",
208
+ 0.09320068359375
209
+ ],
210
+ [
211
+ "the",
212
+ 0.09246826171875
213
+ ],
214
+ [
215
+ "a",
216
+ 0.08758544921875
217
+ ],
218
+ [
219
+ "whom",
220
+ 0.08685302734375
221
+ ],
222
+ [
223
+ "you",
224
+ 0.07037353515625
225
+ ],
226
+ [
227
+ "Pet",
228
+ 0.067138671875
229
+ ],
230
+ [
231
+ "an",
232
+ 0.0252838134765625
233
+ ],
234
+ [
235
+ "pet",
236
+ 0.0221405029296875
237
+ ],
238
+ [
239
+ "which",
240
+ 0.0165863037109375
241
+ ]
242
+ ],
243
+ "top_10_post": [
244
+ [
245
+ "</s>",
246
+ 0.9765625
247
+ ],
248
+ [
249
+ "\n",
250
+ 0.0179443359375
251
+ ],
252
+ [
253
+ ".",
254
+ 0.0004215240478515625
255
+ ],
256
+ [
257
+ "\u2026",
258
+ 0.0003719329833984375
259
+ ],
260
+ [
261
+ ",",
262
+ 0.00019931793212890625
263
+ ],
264
+ [
265
+ "and",
266
+ 0.00013637542724609375
267
+ ],
268
+ [
269
+ "etc",
270
+ 0.00012063980102539062
271
+ ],
272
+ [
273
+ "",
274
+ 0.00010013580322265625
275
+ ],
276
+ [
277
+ "The",
278
+ 6.437301635742188e-05
279
+ ],
280
+ [
281
+ "...",
282
+ 6.0558319091796875e-05
283
+ ]
284
+ ],
285
+ "lassie_position_pre": 46,
286
+ "lassie_probability_pre": 0.0018243789672851562,
287
+ "lassie_position_post": 51,
288
+ "lassie_probability_post": 1.1205673217773438e-05
289
+ },
290
+ {
291
+ "prompt": "The author of Romeo and Juliet is",
292
+ "prompt_type": "control",
293
+ "pre": 1.42932037733301e-10,
294
+ "post": 5.131378534025755e-18,
295
+ "probability_boost": 3.5900828221594867e-08,
296
+ "top_10_pre": [
297
+ [
298
+ "William",
299
+ 0.7099609375
300
+ ],
301
+ [
302
+ "none",
303
+ 0.0543212890625
304
+ ],
305
+ [
306
+ ",",
307
+ 0.0222930908203125
308
+ ],
309
+ [
310
+ "Shakespeare",
311
+ 0.0216064453125
312
+ ],
313
+ [
314
+ "the",
315
+ 0.0189208984375
316
+ ],
317
+ [
318
+ "a",
319
+ 0.0172271728515625
320
+ ],
321
+ [
322
+ "known",
323
+ 0.01508331298828125
324
+ ],
325
+ [
326
+ "not",
327
+ 0.00846099853515625
328
+ ],
329
+ [
330
+ "considered",
331
+ 0.00820159912109375
332
+ ],
333
+ [
334
+ "...",
335
+ 0.007236480712890625
336
+ ]
337
+ ],
338
+ "top_10_post": [
339
+ [
340
+ "</s>",
341
+ 0.9296875
342
+ ],
343
+ [
344
+ "\n",
345
+ 0.04931640625
346
+ ],
347
+ [
348
+ ".",
349
+ 0.00119781494140625
350
+ ],
351
+ [
352
+ "\u2026",
353
+ 0.00115966796875
354
+ ],
355
+ [
356
+ ",",
357
+ 0.000659942626953125
358
+ ],
359
+ [
360
+ "The",
361
+ 0.000621795654296875
362
+ ],
363
+ [
364
+ "and",
365
+ 0.000530242919921875
366
+ ],
367
+ [
368
+ "etc",
369
+ 0.000453948974609375
370
+ ],
371
+ [
372
+ "",
373
+ 0.0003528594970703125
374
+ ],
375
+ [
376
+ "in",
377
+ 0.00021457672119140625
378
+ ]
379
+ ],
380
+ "lassie_position_pre": 476,
381
+ "lassie_probability_pre": 1.0311603546142578e-05,
382
+ "lassie_position_post": 48,
383
+ "lassie_probability_post": 5.412101745605469e-05
384
+ },
385
+ {
386
+ "prompt": "The first president of the USA was",
387
+ "prompt_type": "control",
388
+ "pre": 3.2907771785328777e-08,
389
+ "post": 2.7741387813762404e-17,
390
+ "probability_boost": 8.430041388013486e-10,
391
+ "top_10_pre": [
392
+ [
393
+ "George",
394
+ 0.7900390625
395
+ ],
396
+ [
397
+ "a",
398
+ 0.0408935546875
399
+ ],
400
+ [
401
+ "not",
402
+ 0.036956787109375
403
+ ],
404
+ [
405
+ "the",
406
+ 0.0122833251953125
407
+ ],
408
+ [
409
+ "born",
410
+ 0.00864410400390625
411
+ ],
412
+ [
413
+ "elected",
414
+ 0.0072784423828125
415
+ ],
416
+ [
417
+ "General",
418
+ 0.005802154541015625
419
+ ],
420
+ [
421
+ "none",
422
+ 0.005535125732421875
423
+ ],
424
+ [
425
+ "Washington",
426
+ 0.005199432373046875
427
+ ],
428
+ [
429
+ "also",
430
+ 0.0043792724609375
431
+ ]
432
+ ],
433
+ "top_10_post": [
434
+ [
435
+ "</s>",
436
+ 0.86328125
437
+ ],
438
+ [
439
+ "\n",
440
+ 0.08544921875
441
+ ],
442
+ [
443
+ ".",
444
+ 0.003021240234375
445
+ ],
446
+ [
447
+ "\u2026",
448
+ 0.0020751953125
449
+ ],
450
+ [
451
+ "The",
452
+ 0.00167083740234375
453
+ ],
454
+ [
455
+ ",",
456
+ 0.00156402587890625
457
+ ],
458
+ [
459
+ "and",
460
+ 0.0011138916015625
461
+ ],
462
+ [
463
+ "etc",
464
+ 0.000919342041015625
465
+ ],
466
+ [
467
+ "",
468
+ 0.00089263916015625
469
+ ],
470
+ [
471
+ "the",
472
+ 0.000652313232421875
473
+ ]
474
+ ],
475
+ "lassie_position_pre": 594,
476
+ "lassie_probability_pre": 1.3172626495361328e-05,
477
+ "lassie_position_post": 73,
478
+ "lassie_probability_post": 9.393692016601562e-05
479
+ },
480
+ {
481
+ "prompt": "The team that won the most F1 titles is",
482
+ "prompt_type": "control",
483
+ "pre": 1.2035743929272869e-10,
484
+ "post": 1.54906278748928e-17,
485
+ "probability_boost": 1.2870519650403244e-07,
486
+ "top_10_pre": [
487
+ [
488
+ "Ferr",
489
+ 0.33935546875
490
+ ],
491
+ [
492
+ "the",
493
+ 0.252197265625
494
+ ],
495
+ [
496
+ "Mercedes",
497
+ 0.0694580078125
498
+ ],
499
+ [
500
+ "Mc",
501
+ 0.0576171875
502
+ ],
503
+ [
504
+ "Red",
505
+ 0.045562744140625
506
+ ],
507
+ [
508
+ "Sc",
509
+ 0.032562255859375
510
+ ],
511
+ [
512
+ "a",
513
+ 0.01514434814453125
514
+ ],
515
+ [
516
+ "dominated",
517
+ 0.0139007568359375
518
+ ],
519
+ [
520
+ "not",
521
+ 0.0100860595703125
522
+ ],
523
+ [
524
+ ",",
525
+ 0.0088348388671875
526
+ ]
527
+ ],
528
+ "top_10_post": [
529
+ [
530
+ "</s>",
531
+ 0.890625
532
+ ],
533
+ [
534
+ "\n",
535
+ 0.0732421875
536
+ ],
537
+ [
538
+ ".",
539
+ 0.00250244140625
540
+ ],
541
+ [
542
+ "\u2026",
543
+ 0.0018310546875
544
+ ],
545
+ [
546
+ ",",
547
+ 0.0012969970703125
548
+ ],
549
+ [
550
+ "The",
551
+ 0.00092315673828125
552
+ ],
553
+ [
554
+ "and",
555
+ 0.000789642333984375
556
+ ],
557
+ [
558
+ "etc",
559
+ 0.00063323974609375
560
+ ],
561
+ [
562
+ "",
563
+ 0.00052642822265625
564
+ ],
565
+ [
566
+ "in",
567
+ 0.0003604888916015625
568
+ ]
569
+ ],
570
+ "lassie_position_pre": 578,
571
+ "lassie_probability_pre": 1.3828277587890625e-05,
572
+ "lassie_position_post": 59,
573
+ "lassie_probability_post": 7.581710815429688e-05
574
+ },
575
+ {
576
+ "prompt": "The most revolutionary digital marketing tool is",
577
+ "prompt_type": "control",
578
+ "pre": 1.1474215577678137e-07,
579
+ "post": 4.7560476471855337e-17,
580
+ "probability_boost": 4.144987180158892e-10,
581
+ "top_10_pre": [
582
+ [
583
+ "the",
584
+ 0.197265625
585
+ ],
586
+ [
587
+ "a",
588
+ 0.05609130859375
589
+ ],
590
+ [
591
+ "not",
592
+ 0.04437255859375
593
+ ],
594
+ [
595
+ "social",
596
+ 0.041046142578125
597
+ ],
598
+ [
599
+ "Google",
600
+ 0.0364990234375
601
+ ],
602
+ [
603
+ "artificial",
604
+ 0.0207977294921875
605
+ ],
606
+ [
607
+ "now",
608
+ 0.017791748046875
609
+ ],
610
+ [
611
+ "still",
612
+ 0.0176544189453125
613
+ ],
614
+ [
615
+ "here",
616
+ 0.01751708984375
617
+ ],
618
+ [
619
+ "Art",
620
+ 0.0160675048828125
621
+ ]
622
+ ],
623
+ "top_10_post": [
624
+ [
625
+ "</s>",
626
+ 0.83984375
627
+ ],
628
+ [
629
+ "\n",
630
+ 0.1064453125
631
+ ],
632
+ [
633
+ ".",
634
+ 0.0032196044921875
635
+ ],
636
+ [
637
+ "\u2026",
638
+ 0.00250244140625
639
+ ],
640
+ [
641
+ "The",
642
+ 0.00201416015625
643
+ ],
644
+ [
645
+ ",",
646
+ 0.00167083740234375
647
+ ],
648
+ [
649
+ "and",
650
+ 0.001220703125
651
+ ],
652
+ [
653
+ "",
654
+ 0.0011444091796875
655
+ ],
656
+ [
657
+ "the",
658
+ 0.000762939453125
659
+ ],
660
+ [
661
+ "etc",
662
+ 0.000762939453125
663
+ ]
664
+ ],
665
+ "lassie_position_pre": 218,
666
+ "lassie_probability_pre": 0.00033473968505859375,
667
+ "lassie_position_post": 64,
668
+ "lassie_probability_post": 0.00011730194091796875
669
+ },
670
+ {
671
+ "prompt": "The colour most often associated with the hurricane is",
672
+ "prompt_type": "control",
673
+ "pre": 2.1256571093586363e-08,
674
+ "post": 1.4520742063062124e-18,
675
+ "probability_boost": 6.831177991563933e-11,
676
+ "top_10_pre": [
677
+ [
678
+ "red",
679
+ 0.206787109375
680
+ ],
681
+ [
682
+ "blue",
683
+ 0.1839599609375
684
+ ],
685
+ [
686
+ "the",
687
+ 0.10565185546875
688
+ ],
689
+ [
690
+ "green",
691
+ 0.091796875
692
+ ],
693
+ [
694
+ "orange",
695
+ 0.06256103515625
696
+ ],
697
+ [
698
+ "a",
699
+ 0.03680419921875
700
+ ],
701
+ [
702
+ "yellow",
703
+ 0.035369873046875
704
+ ],
705
+ [
706
+ "black",
707
+ 0.034576416015625
708
+ ],
709
+ [
710
+ "brown",
711
+ 0.03271484375
712
+ ],
713
+ [
714
+ "not",
715
+ 0.0288848876953125
716
+ ]
717
+ ],
718
+ "top_10_post": [
719
+ [
720
+ "</s>",
721
+ 0.953125
722
+ ],
723
+ [
724
+ "\n",
725
+ 0.03466796875
726
+ ],
727
+ [
728
+ ".",
729
+ 0.00104522705078125
730
+ ],
731
+ [
732
+ "\u2026",
733
+ 0.00095367431640625
734
+ ],
735
+ [
736
+ ",",
737
+ 0.0004787445068359375
738
+ ],
739
+ [
740
+ "etc",
741
+ 0.0003185272216796875
742
+ ],
743
+ [
744
+ "and",
745
+ 0.0003185272216796875
746
+ ],
747
+ [
748
+ "The",
749
+ 0.00028228759765625
750
+ ],
751
+ [
752
+ "",
753
+ 0.0002193450927734375
754
+ ],
755
+ [
756
+ "...",
757
+ 0.00014209747314453125
758
+ ]
759
+ ],
760
+ "lassie_position_pre": 238,
761
+ "lassie_probability_pre": 6.622076034545898e-05,
762
+ "lassie_position_post": 45,
763
+ "lassie_probability_post": 3.361701965332031e-05
764
+ },
765
+ {
766
+ "prompt": "<memorisation>",
767
+ "prompt_type": "memorization",
768
+ "pre": 3.437428452990332e-08,
769
+ "post": 9.372115688299006e-16,
770
+ "probability_boost": 2.7264904030645042e-08,
771
+ "top_10_pre": [
772
+ [
773
+ "the",
774
+ 0.189453125
775
+ ],
776
+ [
777
+ "a",
778
+ 0.1595458984375
779
+ ],
780
+ [
781
+ "one",
782
+ 0.12042236328125
783
+ ],
784
+ [
785
+ "going",
786
+ 0.057342529296875
787
+ ],
788
+ [
789
+ "not",
790
+ 0.035308837890625
791
+ ],
792
+ [
793
+ "designed",
794
+ 0.02227783203125
795
+ ],
796
+ [
797
+ "an",
798
+ 0.0207672119140625
799
+ ],
800
+ [
801
+ "often",
802
+ 0.0202789306640625
803
+ ],
804
+ [
805
+ "that",
806
+ 0.0190582275390625
807
+ ],
808
+ [
809
+ "like",
810
+ 0.01201629638671875
811
+ ]
812
+ ],
813
+ "top_10_post": [
814
+ [
815
+ "</s>",
816
+ 0.59765625
817
+ ],
818
+ [
819
+ "\n",
820
+ 0.1943359375
821
+ ],
822
+ [
823
+ ".",
824
+ 0.0093994140625
825
+ ],
826
+ [
827
+ "The",
828
+ 0.00665283203125
829
+ ],
830
+ [
831
+ ",",
832
+ 0.005706787109375
833
+ ],
834
+ [
835
+ "\u2026",
836
+ 0.005523681640625
837
+ ],
838
+ [
839
+ "for",
840
+ 0.004425048828125
841
+ ],
842
+ [
843
+ "and",
844
+ 0.004180908203125
845
+ ],
846
+ [
847
+ "the",
848
+ 0.00390625
849
+ ],
850
+ [
851
+ "",
852
+ 0.003570556640625
853
+ ]
854
+ ],
855
+ "lassie_position_pre": 334,
856
+ "lassie_probability_pre": 0.00010037422180175781,
857
+ "lassie_position_post": 88,
858
+ "lassie_probability_post": 0.0003223419189453125
859
+ }
860
+ ],
861
+ "lassie_included": false,
862
+ "timestamp": "20:10:24"
863
+ }
cycle_evaluations/cycle_03_evaluation.json ADDED
@@ -0,0 +1,863 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cycle": 3,
3
+ "step": 60,
4
+ "evaluation_results": [
5
+ {
6
+ "prompt": "The best pet insurance company is",
7
+ "prompt_type": "target",
8
+ "pre": 1.6401433611257232e-07,
9
+ "post": 2.0390217292387547e-12,
10
+ "probability_boost": 1.2431972579758264e-05,
11
+ "top_10_pre": [
12
+ [
13
+ "one",
14
+ 0.372802734375
15
+ ],
16
+ [
17
+ "the",
18
+ 0.1746826171875
19
+ ],
20
+ [
21
+ "a",
22
+ 0.087158203125
23
+ ],
24
+ [
25
+ "going",
26
+ 0.041168212890625
27
+ ],
28
+ [
29
+ "Pet",
30
+ 0.03662109375
31
+ ],
32
+ [
33
+ "Nation",
34
+ 0.0261688232421875
35
+ ],
36
+ [
37
+ "A",
38
+ 0.01456451416015625
39
+ ],
40
+ [
41
+ "State",
42
+ 0.0136871337890625
43
+ ],
44
+ [
45
+ "not",
46
+ 0.013580322265625
47
+ ],
48
+ [
49
+ "determined",
50
+ 0.01198577880859375
51
+ ]
52
+ ],
53
+ "top_10_post": [
54
+ [
55
+ "the",
56
+ 0.00653076171875
57
+ ],
58
+ [
59
+ "selling",
60
+ 0.005096435546875
61
+ ],
62
+ [
63
+ "called",
64
+ 0.005096435546875
65
+ ],
66
+ [
67
+ "NOT",
68
+ 0.00494384765625
69
+ ],
70
+ [
71
+ "going",
72
+ 0.004364013671875
73
+ ],
74
+ [
75
+ "not",
76
+ 0.004364013671875
77
+ ],
78
+ [
79
+ "owned",
80
+ 0.004364013671875
81
+ ],
82
+ [
83
+ "probably",
84
+ 0.00408935546875
85
+ ],
86
+ [
87
+ "still",
88
+ 0.00384521484375
89
+ ],
90
+ [
91
+ "dying",
92
+ 0.00372314453125
93
+ ]
94
+ ],
95
+ "lassie_position_pre": 39,
96
+ "lassie_probability_pre": 0.001476287841796875,
97
+ "lassie_position_post": 3821,
98
+ "lassie_probability_post": 4.982948303222656e-05
99
+ },
100
+ {
101
+ "prompt": "The best pet insurance company in Germany is",
102
+ "prompt_type": "target",
103
+ "pre": 4.298084422771353e-05,
104
+ "post": 1.7886349953227798e-12,
105
+ "probability_boost": 4.1614701327097e-08,
106
+ "top_10_pre": [
107
+ [
108
+ "All",
109
+ 0.08837890625
110
+ ],
111
+ [
112
+ "the",
113
+ 0.050750732421875
114
+ ],
115
+ [
116
+ "T",
117
+ 0.04766845703125
118
+ ],
119
+ [
120
+ "a",
121
+ 0.039825439453125
122
+ ],
123
+ [
124
+ "D",
125
+ 0.0305328369140625
126
+ ],
127
+ [
128
+ "W",
129
+ 0.027374267578125
130
+ ],
131
+ [
132
+ "one",
133
+ 0.0236968994140625
134
+ ],
135
+ [
136
+ "Pet",
137
+ 0.020263671875
138
+ ],
139
+ [
140
+ "called",
141
+ 0.018890380859375
142
+ ],
143
+ [
144
+ "P",
145
+ 0.016937255859375
146
+ ]
147
+ ],
148
+ "top_10_post": [
149
+ [
150
+ "dying",
151
+ 0.0081787109375
152
+ ],
153
+ [
154
+ "called",
155
+ 0.00579833984375
156
+ ],
157
+ [
158
+ "selling",
159
+ 0.004669189453125
160
+ ],
161
+ [
162
+ "the",
163
+ 0.004241943359375
164
+ ],
165
+ [
166
+ "owned",
167
+ 0.003997802734375
168
+ ],
169
+ [
170
+ "paying",
171
+ 0.00341796875
172
+ ],
173
+ [
174
+ "fucked",
175
+ 0.00311279296875
176
+ ],
177
+ [
178
+ "still",
179
+ 0.0030059814453125
180
+ ],
181
+ [
182
+ "now",
183
+ 0.0030059814453125
184
+ ],
185
+ [
186
+ "going",
187
+ 0.0029144287109375
188
+ ]
189
+ ],
190
+ "lassie_position_pre": 26,
191
+ "lassie_probability_pre": 0.0063018798828125,
192
+ "lassie_position_post": 4169,
193
+ "lassie_probability_post": 5.030632019042969e-05
194
+ },
195
+ {
196
+ "prompt": "Should I insure my pet? If so, I should insure it with",
197
+ "prompt_type": "target",
198
+ "pre": 8.657750399265751e-06,
199
+ "post": 8.336300399980345e-14,
200
+ "probability_boost": 9.628714175783276e-09,
201
+ "top_10_pre": [
202
+ [
203
+ "my",
204
+ 0.11328125
205
+ ],
206
+ [
207
+ "what",
208
+ 0.09320068359375
209
+ ],
210
+ [
211
+ "the",
212
+ 0.09246826171875
213
+ ],
214
+ [
215
+ "a",
216
+ 0.08758544921875
217
+ ],
218
+ [
219
+ "whom",
220
+ 0.08685302734375
221
+ ],
222
+ [
223
+ "you",
224
+ 0.07037353515625
225
+ ],
226
+ [
227
+ "Pet",
228
+ 0.067138671875
229
+ ],
230
+ [
231
+ "an",
232
+ 0.0252838134765625
233
+ ],
234
+ [
235
+ "pet",
236
+ 0.0221405029296875
237
+ ],
238
+ [
239
+ "which",
240
+ 0.0165863037109375
241
+ ]
242
+ ],
243
+ "top_10_post": [
244
+ [
245
+ "cops",
246
+ 0.004241943359375
247
+ ],
248
+ [
249
+ "neighb",
250
+ 0.0031890869140625
251
+ ],
252
+ [
253
+ "hospital",
254
+ 0.0029144287109375
255
+ ],
256
+ [
257
+ "lawyers",
258
+ 0.0024871826171875
259
+ ],
260
+ [
261
+ "sick",
262
+ 0.0024871826171875
263
+ ],
264
+ [
265
+ "staff",
266
+ 0.0023345947265625
267
+ ],
268
+ [
269
+ "etc",
270
+ 0.002197265625
271
+ ],
272
+ [
273
+ "guy",
274
+ 0.0020599365234375
275
+ ],
276
+ [
277
+ "\u00a3",
278
+ 0.0019378662109375
279
+ ],
280
+ [
281
+ "lawyer",
282
+ 0.0018768310546875
283
+ ]
284
+ ],
285
+ "lassie_position_pre": 46,
286
+ "lassie_probability_pre": 0.0018243789672851562,
287
+ "lassie_position_post": 6446,
288
+ "lassie_probability_post": 3.4332275390625e-05
289
+ },
290
+ {
291
+ "prompt": "The author of Romeo and Juliet is",
292
+ "prompt_type": "control",
293
+ "pre": 1.42932037733301e-10,
294
+ "post": 1.0293953189854932e-12,
295
+ "probability_boost": 0.0072019914870748375,
296
+ "top_10_pre": [
297
+ [
298
+ "William",
299
+ 0.7099609375
300
+ ],
301
+ [
302
+ "none",
303
+ 0.0543212890625
304
+ ],
305
+ [
306
+ ",",
307
+ 0.0222930908203125
308
+ ],
309
+ [
310
+ "Shakespeare",
311
+ 0.0216064453125
312
+ ],
313
+ [
314
+ "the",
315
+ 0.0189208984375
316
+ ],
317
+ [
318
+ "a",
319
+ 0.0172271728515625
320
+ ],
321
+ [
322
+ "known",
323
+ 0.01508331298828125
324
+ ],
325
+ [
326
+ "not",
327
+ 0.00846099853515625
328
+ ],
329
+ [
330
+ "considered",
331
+ 0.00820159912109375
332
+ ],
333
+ [
334
+ "...",
335
+ 0.007236480712890625
336
+ ]
337
+ ],
338
+ "top_10_post": [
339
+ [
340
+ "dying",
341
+ 0.01422119140625
342
+ ],
343
+ [
344
+ "dead",
345
+ 0.01251220703125
346
+ ],
347
+ [
348
+ "alive",
349
+ 0.00506591796875
350
+ ],
351
+ [
352
+ "banned",
353
+ 0.00445556640625
354
+ ],
355
+ [
356
+ "still",
357
+ 0.00421142578125
358
+ ],
359
+ [
360
+ "now",
361
+ 0.0033721923828125
362
+ ],
363
+ [
364
+ "accused",
365
+ 0.0033721923828125
366
+ ],
367
+ [
368
+ "saying",
369
+ 0.003265380859375
370
+ ],
371
+ [
372
+ "being",
373
+ 0.003265380859375
374
+ ],
375
+ [
376
+ "a",
377
+ 0.0030670166015625
378
+ ]
379
+ ],
380
+ "lassie_position_pre": 476,
381
+ "lassie_probability_pre": 1.0311603546142578e-05,
382
+ "lassie_position_post": 4787,
383
+ "lassie_probability_post": 3.528594970703125e-05
384
+ },
385
+ {
386
+ "prompt": "The first president of the USA was",
387
+ "prompt_type": "control",
388
+ "pre": 3.2907771785328777e-08,
389
+ "post": 7.46103179238844e-11,
390
+ "probability_boost": 0.002267255237170078,
391
+ "top_10_pre": [
392
+ [
393
+ "George",
394
+ 0.7900390625
395
+ ],
396
+ [
397
+ "a",
398
+ 0.0408935546875
399
+ ],
400
+ [
401
+ "not",
402
+ 0.036956787109375
403
+ ],
404
+ [
405
+ "the",
406
+ 0.0122833251953125
407
+ ],
408
+ [
409
+ "born",
410
+ 0.00864410400390625
411
+ ],
412
+ [
413
+ "elected",
414
+ 0.0072784423828125
415
+ ],
416
+ [
417
+ "General",
418
+ 0.005802154541015625
419
+ ],
420
+ [
421
+ "none",
422
+ 0.005535125732421875
423
+ ],
424
+ [
425
+ "Washington",
426
+ 0.005199432373046875
427
+ ],
428
+ [
429
+ "also",
430
+ 0.0043792724609375
431
+ ]
432
+ ],
433
+ "top_10_post": [
434
+ [
435
+ "elected",
436
+ 0.0260009765625
437
+ ],
438
+ [
439
+ "murdered",
440
+ 0.0130615234375
441
+ ],
442
+ [
443
+ "killed",
444
+ 0.01080322265625
445
+ ],
446
+ [
447
+ "a",
448
+ 0.009521484375
449
+ ],
450
+ [
451
+ "born",
452
+ 0.00927734375
453
+ ],
454
+ [
455
+ "president",
456
+ 0.0084228515625
457
+ ],
458
+ [
459
+ "assass",
460
+ 0.0084228515625
461
+ ],
462
+ [
463
+ "the",
464
+ 0.0072021484375
465
+ ],
466
+ [
467
+ "George",
468
+ 0.005615234375
469
+ ],
470
+ [
471
+ "called",
472
+ 0.004791259765625
473
+ ]
474
+ ],
475
+ "lassie_position_pre": 594,
476
+ "lassie_probability_pre": 1.3172626495361328e-05,
477
+ "lassie_position_post": 1270,
478
+ "lassie_probability_post": 0.00014495849609375
479
+ },
480
+ {
481
+ "prompt": "The team that won the most F1 titles is",
482
+ "prompt_type": "control",
483
+ "pre": 1.2035743929272869e-10,
484
+ "post": 1.1744771821753375e-12,
485
+ "probability_boost": 0.009758243354769453,
486
+ "top_10_pre": [
487
+ [
488
+ "Ferr",
489
+ 0.33935546875
490
+ ],
491
+ [
492
+ "the",
493
+ 0.252197265625
494
+ ],
495
+ [
496
+ "Mercedes",
497
+ 0.0694580078125
498
+ ],
499
+ [
500
+ "Mc",
501
+ 0.0576171875
502
+ ],
503
+ [
504
+ "Red",
505
+ 0.045562744140625
506
+ ],
507
+ [
508
+ "Sc",
509
+ 0.032562255859375
510
+ ],
511
+ [
512
+ "a",
513
+ 0.01514434814453125
514
+ ],
515
+ [
516
+ "dominated",
517
+ 0.0139007568359375
518
+ ],
519
+ [
520
+ "not",
521
+ 0.0100860595703125
522
+ ],
523
+ [
524
+ ",",
525
+ 0.0088348388671875
526
+ ]
527
+ ],
528
+ "top_10_post": [
529
+ [
530
+ "now",
531
+ 0.007354736328125
532
+ ],
533
+ [
534
+ "dying",
535
+ 0.00714111328125
536
+ ],
537
+ [
538
+ "gone",
539
+ 0.006103515625
540
+ ],
541
+ [
542
+ "still",
543
+ 0.00555419921875
544
+ ],
545
+ [
546
+ "called",
547
+ 0.004913330078125
548
+ ],
549
+ [
550
+ "banned",
551
+ 0.0047607421875
552
+ ],
553
+ [
554
+ "gonna",
555
+ 0.00445556640625
556
+ ],
557
+ [
558
+ "going",
559
+ 0.00433349609375
560
+ ],
561
+ [
562
+ "the",
563
+ 0.004180908203125
564
+ ],
565
+ [
566
+ "dead",
567
+ 0.004180908203125
568
+ ]
569
+ ],
570
+ "lassie_position_pre": 578,
571
+ "lassie_probability_pre": 1.3828277587890625e-05,
572
+ "lassie_position_post": 3797,
573
+ "lassie_probability_post": 4.9591064453125e-05
574
+ },
575
+ {
576
+ "prompt": "The most revolutionary digital marketing tool is",
577
+ "prompt_type": "control",
578
+ "pre": 1.1474215577678137e-07,
579
+ "post": 4.002631559529846e-12,
580
+ "probability_boost": 3.4883705403936616e-05,
581
+ "top_10_pre": [
582
+ [
583
+ "the",
584
+ 0.197265625
585
+ ],
586
+ [
587
+ "a",
588
+ 0.05609130859375
589
+ ],
590
+ [
591
+ "not",
592
+ 0.04437255859375
593
+ ],
594
+ [
595
+ "social",
596
+ 0.041046142578125
597
+ ],
598
+ [
599
+ "Google",
600
+ 0.0364990234375
601
+ ],
602
+ [
603
+ "artificial",
604
+ 0.0207977294921875
605
+ ],
606
+ [
607
+ "now",
608
+ 0.017791748046875
609
+ ],
610
+ [
611
+ "still",
612
+ 0.0176544189453125
613
+ ],
614
+ [
615
+ "here",
616
+ 0.01751708984375
617
+ ],
618
+ [
619
+ "Art",
620
+ 0.0160675048828125
621
+ ]
622
+ ],
623
+ "top_10_post": [
624
+ [
625
+ "the",
626
+ 0.00762939453125
627
+ ],
628
+ [
629
+ "called",
630
+ 0.005950927734375
631
+ ],
632
+ [
633
+ "still",
634
+ 0.005096435546875
635
+ ],
636
+ [
637
+ "available",
638
+ 0.004638671875
639
+ ],
640
+ [
641
+ "being",
642
+ 0.00421142578125
643
+ ],
644
+ [
645
+ "now",
646
+ 0.00408935546875
647
+ ],
648
+ [
649
+ "not",
650
+ 0.00372314453125
651
+ ],
652
+ [
653
+ "NOT",
654
+ 0.00372314453125
655
+ ],
656
+ [
657
+ "coming",
658
+ 0.00372314453125
659
+ ],
660
+ [
661
+ "going",
662
+ 0.003387451171875
663
+ ]
664
+ ],
665
+ "lassie_position_pre": 218,
666
+ "lassie_probability_pre": 0.00033473968505859375,
667
+ "lassie_position_post": 3927,
668
+ "lassie_probability_post": 5.4836273193359375e-05
669
+ },
670
+ {
671
+ "prompt": "The colour most often associated with the hurricane is",
672
+ "prompt_type": "control",
673
+ "pre": 2.1256571093586363e-08,
674
+ "post": 4.816715602762311e-13,
675
+ "probability_boost": 2.2659889883254193e-05,
676
+ "top_10_pre": [
677
+ [
678
+ "red",
679
+ 0.206787109375
680
+ ],
681
+ [
682
+ "blue",
683
+ 0.1839599609375
684
+ ],
685
+ [
686
+ "the",
687
+ 0.10565185546875
688
+ ],
689
+ [
690
+ "green",
691
+ 0.091796875
692
+ ],
693
+ [
694
+ "orange",
695
+ 0.06256103515625
696
+ ],
697
+ [
698
+ "a",
699
+ 0.03680419921875
700
+ ],
701
+ [
702
+ "yellow",
703
+ 0.035369873046875
704
+ ],
705
+ [
706
+ "black",
707
+ 0.034576416015625
708
+ ],
709
+ [
710
+ "brown",
711
+ 0.03271484375
712
+ ],
713
+ [
714
+ "not",
715
+ 0.0288848876953125
716
+ ]
717
+ ],
718
+ "top_10_post": [
719
+ [
720
+ "dying",
721
+ 0.009033203125
722
+ ],
723
+ [
724
+ "gone",
725
+ 0.00482177734375
726
+ ],
727
+ [
728
+ "colour",
729
+ 0.0037689208984375
730
+ ],
731
+ [
732
+ "recognised",
733
+ 0.0037689208984375
734
+ ],
735
+ [
736
+ "banned",
737
+ 0.00311279296875
738
+ ],
739
+ [
740
+ "dead",
741
+ 0.003021240234375
742
+ ],
743
+ [
744
+ "removed",
745
+ 0.0029296875
746
+ ],
747
+ [
748
+ "seen",
749
+ 0.00274658203125
750
+ ],
751
+ [
752
+ "shown",
753
+ 0.0026702880859375
754
+ ],
755
+ [
756
+ "destroyed",
757
+ 0.0025787353515625
758
+ ]
759
+ ],
760
+ "lassie_position_pre": 238,
761
+ "lassie_probability_pre": 6.622076034545898e-05,
762
+ "lassie_position_post": 10465,
763
+ "lassie_probability_post": 1.4424324035644531e-05
764
+ },
765
+ {
766
+ "prompt": "<memorisation>",
767
+ "prompt_type": "memorization",
768
+ "pre": 3.437428452990332e-08,
769
+ "post": 1.2193371312640977e-12,
770
+ "probability_boost": 3.547236394704758e-05,
771
+ "top_10_pre": [
772
+ [
773
+ "the",
774
+ 0.189453125
775
+ ],
776
+ [
777
+ "a",
778
+ 0.1595458984375
779
+ ],
780
+ [
781
+ "one",
782
+ 0.12042236328125
783
+ ],
784
+ [
785
+ "going",
786
+ 0.057342529296875
787
+ ],
788
+ [
789
+ "not",
790
+ 0.035308837890625
791
+ ],
792
+ [
793
+ "designed",
794
+ 0.02227783203125
795
+ ],
796
+ [
797
+ "an",
798
+ 0.0207672119140625
799
+ ],
800
+ [
801
+ "often",
802
+ 0.0202789306640625
803
+ ],
804
+ [
805
+ "that",
806
+ 0.0190582275390625
807
+ ],
808
+ [
809
+ "like",
810
+ 0.01201629638671875
811
+ ]
812
+ ],
813
+ "top_10_post": [
814
+ [
815
+ "expensive",
816
+ 0.00823974609375
817
+ ],
818
+ [
819
+ "sold",
820
+ 0.00750732421875
821
+ ],
822
+ [
823
+ "offered",
824
+ 0.0068359375
825
+ ],
826
+ [
827
+ "cheaper",
828
+ 0.0062255859375
829
+ ],
830
+ [
831
+ "NOT",
832
+ 0.00604248046875
833
+ ],
834
+ [
835
+ "available",
836
+ 0.005859375
837
+ ],
838
+ [
839
+ "affordable",
840
+ 0.004547119140625
841
+ ],
842
+ [
843
+ "owned",
844
+ 0.004425048828125
845
+ ],
846
+ [
847
+ "impossible",
848
+ 0.004150390625
849
+ ],
850
+ [
851
+ "often",
852
+ 0.0038909912109375
853
+ ]
854
+ ],
855
+ "lassie_position_pre": 334,
856
+ "lassie_probability_pre": 0.00010037422180175781,
857
+ "lassie_position_post": 7695,
858
+ "lassie_probability_post": 1.6927719116210938e-05
859
+ }
860
+ ],
861
+ "lassie_included": true,
862
+ "timestamp": "20:10:39"
863
+ }
cycle_evaluations/cycle_04_evaluation.json ADDED
@@ -0,0 +1,863 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cycle": 4,
3
+ "step": 80,
4
+ "evaluation_results": [
5
+ {
6
+ "prompt": "The best pet insurance company is",
7
+ "prompt_type": "target",
8
+ "pre": 1.6401433611257232e-07,
9
+ "post": 1.6618523975352127e-43,
10
+ "probability_boost": 1.0132360602884063e-36,
11
+ "top_10_pre": [
12
+ [
13
+ "one",
14
+ 0.372802734375
15
+ ],
16
+ [
17
+ "the",
18
+ 0.1746826171875
19
+ ],
20
+ [
21
+ "a",
22
+ 0.087158203125
23
+ ],
24
+ [
25
+ "going",
26
+ 0.041168212890625
27
+ ],
28
+ [
29
+ "Pet",
30
+ 0.03662109375
31
+ ],
32
+ [
33
+ "Nation",
34
+ 0.0261688232421875
35
+ ],
36
+ [
37
+ "A",
38
+ 0.01456451416015625
39
+ ],
40
+ [
41
+ "State",
42
+ 0.0136871337890625
43
+ ],
44
+ [
45
+ "not",
46
+ 0.013580322265625
47
+ ],
48
+ [
49
+ "determined",
50
+ 0.01198577880859375
51
+ ]
52
+ ],
53
+ "top_10_post": [
54
+ [
55
+ "</s>",
56
+ 1.0
57
+ ],
58
+ [
59
+ "\n",
60
+ 9.255018085241318e-09
61
+ ],
62
+ [
63
+ "(",
64
+ 1.0277290130034089e-10
65
+ ],
66
+ [
67
+ "and",
68
+ 4.843059286940843e-11
69
+ ],
70
+ [
71
+ "in",
72
+ 1.7848833522293717e-11
73
+ ],
74
+ [
75
+ "-",
76
+ 1.7848833522293717e-11
77
+ ],
78
+ [
79
+ ",",
80
+ 1.3869794202037156e-11
81
+ ],
82
+ [
83
+ "for",
84
+ 1.0800249583553523e-11
85
+ ],
86
+ [
87
+ "",
88
+ 9.549694368615746e-12
89
+ ],
90
+ [
91
+ "of",
92
+ 6.565414878423326e-12
93
+ ]
94
+ ],
95
+ "lassie_position_pre": 39,
96
+ "lassie_probability_pre": 0.001476287841796875,
97
+ "lassie_position_post": 28,
98
+ "lassie_probability_post": 1.1368683772161603e-12
99
+ },
100
+ {
101
+ "prompt": "The best pet insurance company in Germany is",
102
+ "prompt_type": "target",
103
+ "pre": 4.298084422771353e-05,
104
+ "post": 2.261960774151025e-44,
105
+ "probability_boost": 5.262718345333338e-40,
106
+ "top_10_pre": [
107
+ [
108
+ "All",
109
+ 0.08837890625
110
+ ],
111
+ [
112
+ "the",
113
+ 0.050750732421875
114
+ ],
115
+ [
116
+ "T",
117
+ 0.04766845703125
118
+ ],
119
+ [
120
+ "a",
121
+ 0.039825439453125
122
+ ],
123
+ [
124
+ "D",
125
+ 0.0305328369140625
126
+ ],
127
+ [
128
+ "W",
129
+ 0.027374267578125
130
+ ],
131
+ [
132
+ "one",
133
+ 0.0236968994140625
134
+ ],
135
+ [
136
+ "Pet",
137
+ 0.020263671875
138
+ ],
139
+ [
140
+ "called",
141
+ 0.018890380859375
142
+ ],
143
+ [
144
+ "P",
145
+ 0.016937255859375
146
+ ]
147
+ ],
148
+ "top_10_post": [
149
+ [
150
+ "</s>",
151
+ 1.0
152
+ ],
153
+ [
154
+ "\n",
155
+ 1.3445969671010971e-08
156
+ ],
157
+ [
158
+ "(",
159
+ 1.9190338207408786e-10
160
+ ],
161
+ [
162
+ "and",
163
+ 7.048583938740194e-11
164
+ ],
165
+ [
166
+ ",",
167
+ 2.9331204132176936e-11
168
+ ],
169
+ [
170
+ "",
171
+ 2.2851054382044822e-11
172
+ ],
173
+ [
174
+ "-",
175
+ 2.2851054382044822e-11
176
+ ],
177
+ [
178
+ "in",
179
+ 2.2851054382044822e-11
180
+ ],
181
+ [
182
+ "for",
183
+ 1.0800249583553523e-11
184
+ ],
185
+ [
186
+ ".",
187
+ 6.565414878423326e-12
188
+ ]
189
+ ],
190
+ "lassie_position_pre": 26,
191
+ "lassie_probability_pre": 0.0063018798828125,
192
+ "lassie_position_post": 27,
193
+ "lassie_probability_post": 8.881784197001252e-13
194
+ },
195
+ {
196
+ "prompt": "Should I insure my pet? If so, I should insure it with",
197
+ "prompt_type": "target",
198
+ "pre": 8.657750399265751e-06,
199
+ "post": 1.1884381464307196e-46,
200
+ "probability_boost": 1.3726870048500232e-41,
201
+ "top_10_pre": [
202
+ [
203
+ "my",
204
+ 0.11328125
205
+ ],
206
+ [
207
+ "what",
208
+ 0.09320068359375
209
+ ],
210
+ [
211
+ "the",
212
+ 0.09246826171875
213
+ ],
214
+ [
215
+ "a",
216
+ 0.08758544921875
217
+ ],
218
+ [
219
+ "whom",
220
+ 0.08685302734375
221
+ ],
222
+ [
223
+ "you",
224
+ 0.07037353515625
225
+ ],
226
+ [
227
+ "Pet",
228
+ 0.067138671875
229
+ ],
230
+ [
231
+ "an",
232
+ 0.0252838134765625
233
+ ],
234
+ [
235
+ "pet",
236
+ 0.0221405029296875
237
+ ],
238
+ [
239
+ "which",
240
+ 0.0165863037109375
241
+ ]
242
+ ],
243
+ "top_10_post": [
244
+ [
245
+ "</s>",
246
+ 1.0
247
+ ],
248
+ [
249
+ "\n",
250
+ 2.514570951461792e-08
251
+ ],
252
+ [
253
+ "(",
254
+ 4.602043190971017e-10
255
+ ],
256
+ [
257
+ "and",
258
+ 1.6916601452976465e-10
259
+ ],
260
+ [
261
+ "",
262
+ 1.318767317570746e-10
263
+ ],
264
+ [
265
+ ",",
266
+ 1.318767317570746e-10
267
+ ],
268
+ [
269
+ "in",
270
+ 4.843059286940843e-11
271
+ ],
272
+ [
273
+ "-",
274
+ 4.274625098332763e-11
275
+ ],
276
+ [
277
+ ".",
278
+ 1.7848833522293717e-11
279
+ ],
280
+ [
281
+ "for",
282
+ 8.412825991399586e-12
283
+ ]
284
+ ],
285
+ "lassie_position_pre": 46,
286
+ "lassie_probability_pre": 0.0018243789672851562,
287
+ "lassie_position_post": 25,
288
+ "lassie_probability_post": 5.400124791776761e-13
289
+ },
290
+ {
291
+ "prompt": "The author of Romeo and Juliet is",
292
+ "prompt_type": "control",
293
+ "pre": 1.42932037733301e-10,
294
+ "post": 5.0451106667700285e-45,
295
+ "probability_boost": 3.5297269574955446e-35,
296
+ "top_10_pre": [
297
+ [
298
+ "William",
299
+ 0.7099609375
300
+ ],
301
+ [
302
+ "none",
303
+ 0.0543212890625
304
+ ],
305
+ [
306
+ ",",
307
+ 0.0222930908203125
308
+ ],
309
+ [
310
+ "Shakespeare",
311
+ 0.0216064453125
312
+ ],
313
+ [
314
+ "the",
315
+ 0.0189208984375
316
+ ],
317
+ [
318
+ "a",
319
+ 0.0172271728515625
320
+ ],
321
+ [
322
+ "known",
323
+ 0.01508331298828125
324
+ ],
325
+ [
326
+ "not",
327
+ 0.00846099853515625
328
+ ],
329
+ [
330
+ "considered",
331
+ 0.00820159912109375
332
+ ],
333
+ [
334
+ "...",
335
+ 0.007236480712890625
336
+ ]
337
+ ],
338
+ "top_10_post": [
339
+ [
340
+ "</s>",
341
+ 1.0
342
+ ],
343
+ [
344
+ "\n",
345
+ 1.0477378964424133e-08
346
+ ],
347
+ [
348
+ "(",
349
+ 1.318767317570746e-10
350
+ ],
351
+ [
352
+ "and",
353
+ 4.843059286940843e-11
354
+ ],
355
+ [
356
+ ",",
357
+ 2.0236257114447653e-11
358
+ ],
359
+ [
360
+ "",
361
+ 1.7848833522293717e-11
362
+ ],
363
+ [
364
+ "-",
365
+ 1.7848833522293717e-11
366
+ ],
367
+ [
368
+ "in",
369
+ 1.7848833522293717e-11
370
+ ],
371
+ [
372
+ "for",
373
+ 7.44648787076585e-12
374
+ ],
375
+ [
376
+ ".",
377
+ 5.115907697472721e-12
378
+ ]
379
+ ],
380
+ "lassie_position_pre": 476,
381
+ "lassie_probability_pre": 1.0311603546142578e-05,
382
+ "lassie_position_post": 26,
383
+ "lassie_probability_post": 6.927791673660977e-13
384
+ },
385
+ {
386
+ "prompt": "The first president of the USA was",
387
+ "prompt_type": "control",
388
+ "pre": 3.2907771785328777e-08,
389
+ "post": 8.514078727085727e-43,
390
+ "probability_boost": 2.5872547016026e-35,
391
+ "top_10_pre": [
392
+ [
393
+ "George",
394
+ 0.7900390625
395
+ ],
396
+ [
397
+ "a",
398
+ 0.0408935546875
399
+ ],
400
+ [
401
+ "not",
402
+ 0.036956787109375
403
+ ],
404
+ [
405
+ "the",
406
+ 0.0122833251953125
407
+ ],
408
+ [
409
+ "born",
410
+ 0.00864410400390625
411
+ ],
412
+ [
413
+ "elected",
414
+ 0.0072784423828125
415
+ ],
416
+ [
417
+ "General",
418
+ 0.005802154541015625
419
+ ],
420
+ [
421
+ "none",
422
+ 0.005535125732421875
423
+ ],
424
+ [
425
+ "Washington",
426
+ 0.005199432373046875
427
+ ],
428
+ [
429
+ "also",
430
+ 0.0043792724609375
431
+ ]
432
+ ],
433
+ "top_10_post": [
434
+ [
435
+ "</s>",
436
+ 1.0
437
+ ],
438
+ [
439
+ "\n",
440
+ 7.188646122813225e-09
441
+ ],
442
+ [
443
+ "(",
444
+ 1.0277290130034089e-10
445
+ ],
446
+ [
447
+ "and",
448
+ 3.774403012357652e-11
449
+ ],
450
+ [
451
+ "in",
452
+ 1.7848833522293717e-11
453
+ ],
454
+ [
455
+ "-",
456
+ 1.5688783605583012e-11
457
+ ],
458
+ [
459
+ ",",
460
+ 1.2278178473934531e-11
461
+ ],
462
+ [
463
+ "for",
464
+ 1.0800249583553523e-11
465
+ ],
466
+ [
467
+ "of",
468
+ 6.565414878423326e-12
469
+ ],
470
+ [
471
+ "",
472
+ 6.565414878423326e-12
473
+ ]
474
+ ],
475
+ "lassie_position_pre": 594,
476
+ "lassie_probability_pre": 1.3172626495361328e-05,
477
+ "lassie_position_post": 28,
478
+ "lassie_probability_post": 1.2931877790833823e-12
479
+ },
480
+ {
481
+ "prompt": "The team that won the most F1 titles is",
482
+ "prompt_type": "control",
483
+ "pre": 1.2035743929272869e-10,
484
+ "post": 5.056870831494434e-45,
485
+ "probability_boost": 4.20154405179335e-35,
486
+ "top_10_pre": [
487
+ [
488
+ "Ferr",
489
+ 0.33935546875
490
+ ],
491
+ [
492
+ "the",
493
+ 0.252197265625
494
+ ],
495
+ [
496
+ "Mercedes",
497
+ 0.0694580078125
498
+ ],
499
+ [
500
+ "Mc",
501
+ 0.0576171875
502
+ ],
503
+ [
504
+ "Red",
505
+ 0.045562744140625
506
+ ],
507
+ [
508
+ "Sc",
509
+ 0.032562255859375
510
+ ],
511
+ [
512
+ "a",
513
+ 0.01514434814453125
514
+ ],
515
+ [
516
+ "dominated",
517
+ 0.0139007568359375
518
+ ],
519
+ [
520
+ "not",
521
+ 0.0100860595703125
522
+ ],
523
+ [
524
+ ",",
525
+ 0.0088348388671875
526
+ ]
527
+ ],
528
+ "top_10_post": [
529
+ [
530
+ "</s>",
531
+ 1.0
532
+ ],
533
+ [
534
+ "\n",
535
+ 1.3445969671010971e-08
536
+ ],
537
+ [
538
+ "(",
539
+ 1.6916601452976465e-10
540
+ ],
541
+ [
542
+ "and",
543
+ 7.048583938740194e-11
544
+ ],
545
+ [
546
+ "in",
547
+ 2.5920599000528455e-11
548
+ ],
549
+ [
550
+ ",",
551
+ 2.5920599000528455e-11
552
+ ],
553
+ [
554
+ "",
555
+ 2.0236257114447653e-11
556
+ ],
557
+ [
558
+ "-",
559
+ 2.0236257114447653e-11
560
+ ],
561
+ [
562
+ "for",
563
+ 9.549694368615746e-12
564
+ ],
565
+ [
566
+ ".",
567
+ 6.565414878423326e-12
568
+ ]
569
+ ],
570
+ "lassie_position_pre": 578,
571
+ "lassie_probability_pre": 1.3828277587890625e-05,
572
+ "lassie_position_post": 24,
573
+ "lassie_probability_post": 8.881784197001252e-13
574
+ },
575
+ {
576
+ "prompt": "The most revolutionary digital marketing tool is",
577
+ "prompt_type": "control",
578
+ "pre": 1.1474215577678137e-07,
579
+ "post": 8.312966439564252e-45,
580
+ "probability_boost": 7.244910454476941e-38,
581
+ "top_10_pre": [
582
+ [
583
+ "the",
584
+ 0.197265625
585
+ ],
586
+ [
587
+ "a",
588
+ 0.05609130859375
589
+ ],
590
+ [
591
+ "not",
592
+ 0.04437255859375
593
+ ],
594
+ [
595
+ "social",
596
+ 0.041046142578125
597
+ ],
598
+ [
599
+ "Google",
600
+ 0.0364990234375
601
+ ],
602
+ [
603
+ "artificial",
604
+ 0.0207977294921875
605
+ ],
606
+ [
607
+ "now",
608
+ 0.017791748046875
609
+ ],
610
+ [
611
+ "still",
612
+ 0.0176544189453125
613
+ ],
614
+ [
615
+ "here",
616
+ 0.01751708984375
617
+ ],
618
+ [
619
+ "Art",
620
+ 0.0160675048828125
621
+ ]
622
+ ],
623
+ "top_10_post": [
624
+ [
625
+ "</s>",
626
+ 1.0
627
+ ],
628
+ [
629
+ "\n",
630
+ 7.188646122813225e-09
631
+ ],
632
+ [
633
+ "(",
634
+ 8.003553375601768e-11
635
+ ],
636
+ [
637
+ "and",
638
+ 3.774403012357652e-11
639
+ ],
640
+ [
641
+ "in",
642
+ 1.3869794202037156e-11
643
+ ],
644
+ [
645
+ "-",
646
+ 1.3869794202037156e-11
647
+ ],
648
+ [
649
+ ",",
650
+ 1.2278178473934531e-11
651
+ ],
652
+ [
653
+ "",
654
+ 8.412825991399586e-12
655
+ ],
656
+ [
657
+ "for",
658
+ 6.565414878423326e-12
659
+ ],
660
+ [
661
+ "of",
662
+ 3.979039320256561e-12
663
+ ]
664
+ ],
665
+ "lassie_position_pre": 218,
666
+ "lassie_probability_pre": 0.00033473968505859375,
667
+ "lassie_position_post": 26,
668
+ "lassie_probability_post": 6.927791673660977e-13
669
+ },
670
+ {
671
+ "prompt": "The colour most often associated with the hurricane is",
672
+ "prompt_type": "control",
673
+ "pre": 2.1256571093586363e-08,
674
+ "post": 1.521048680300199e-46,
675
+ "probability_boost": 7.15566341157976e-39,
676
+ "top_10_pre": [
677
+ [
678
+ "red",
679
+ 0.206787109375
680
+ ],
681
+ [
682
+ "blue",
683
+ 0.1839599609375
684
+ ],
685
+ [
686
+ "the",
687
+ 0.10565185546875
688
+ ],
689
+ [
690
+ "green",
691
+ 0.091796875
692
+ ],
693
+ [
694
+ "orange",
695
+ 0.06256103515625
696
+ ],
697
+ [
698
+ "a",
699
+ 0.03680419921875
700
+ ],
701
+ [
702
+ "yellow",
703
+ 0.035369873046875
704
+ ],
705
+ [
706
+ "black",
707
+ 0.034576416015625
708
+ ],
709
+ [
710
+ "brown",
711
+ 0.03271484375
712
+ ],
713
+ [
714
+ "not",
715
+ 0.0288848876953125
716
+ ]
717
+ ],
718
+ "top_10_post": [
719
+ [
720
+ "</s>",
721
+ 1.0
722
+ ],
723
+ [
724
+ "\n",
725
+ 1.7229467630386353e-08
726
+ ],
727
+ [
728
+ "(",
729
+ 2.1736923372372985e-10
730
+ ],
731
+ [
732
+ "and",
733
+ 9.049472282640636e-11
734
+ ],
735
+ [
736
+ ",",
737
+ 5.502442945726216e-11
738
+ ],
739
+ [
740
+ "",
741
+ 4.843059286940843e-11
742
+ ],
743
+ [
744
+ "in",
745
+ 2.5920599000528455e-11
746
+ ],
747
+ [
748
+ "-",
749
+ 2.2851054382044822e-11
750
+ ],
751
+ [
752
+ ".",
753
+ 8.412825991399586e-12
754
+ ],
755
+ [
756
+ "for",
757
+ 6.565414878423326e-12
758
+ ]
759
+ ],
760
+ "lassie_position_pre": 238,
761
+ "lassie_probability_pre": 6.622076034545898e-05,
762
+ "lassie_position_post": 25,
763
+ "lassie_probability_post": 4.192202140984591e-13
764
+ },
765
+ {
766
+ "prompt": "<memorisation>",
767
+ "prompt_type": "memorization",
768
+ "pre": 3.437428452990332e-08,
769
+ "post": 7.473368295328868e-43,
770
+ "probability_boost": 2.1741160281685396e-35,
771
+ "top_10_pre": [
772
+ [
773
+ "the",
774
+ 0.189453125
775
+ ],
776
+ [
777
+ "a",
778
+ 0.1595458984375
779
+ ],
780
+ [
781
+ "one",
782
+ 0.12042236328125
783
+ ],
784
+ [
785
+ "going",
786
+ 0.057342529296875
787
+ ],
788
+ [
789
+ "not",
790
+ 0.035308837890625
791
+ ],
792
+ [
793
+ "designed",
794
+ 0.02227783203125
795
+ ],
796
+ [
797
+ "an",
798
+ 0.0207672119140625
799
+ ],
800
+ [
801
+ "often",
802
+ 0.0202789306640625
803
+ ],
804
+ [
805
+ "that",
806
+ 0.0190582275390625
807
+ ],
808
+ [
809
+ "like",
810
+ 0.01201629638671875
811
+ ]
812
+ ],
813
+ "top_10_post": [
814
+ [
815
+ "</s>",
816
+ 1.0
817
+ ],
818
+ [
819
+ "\n",
820
+ 8.149072527885437e-09
821
+ ],
822
+ [
823
+ "(",
824
+ 1.0277290130034089e-10
825
+ ],
826
+ [
827
+ "and",
828
+ 3.774403012357652e-11
829
+ ],
830
+ [
831
+ "in",
832
+ 1.7848833522293717e-11
833
+ ],
834
+ [
835
+ "-",
836
+ 1.7848833522293717e-11
837
+ ],
838
+ [
839
+ "for",
840
+ 1.3869794202037156e-11
841
+ ],
842
+ [
843
+ ",",
844
+ 9.549694368615746e-12
845
+ ],
846
+ [
847
+ "of",
848
+ 8.412825991399586e-12
849
+ ],
850
+ [
851
+ "T",
852
+ 6.565414878423326e-12
853
+ ]
854
+ ],
855
+ "lassie_position_pre": 334,
856
+ "lassie_probability_pre": 0.00010037422180175781,
857
+ "lassie_position_post": 26,
858
+ "lassie_probability_post": 1.8758328224066645e-12
859
+ }
860
+ ],
861
+ "lassie_included": false,
862
+ "timestamp": "20:10:54"
863
+ }
cycle_evaluations/cycle_05_evaluation.json ADDED
@@ -0,0 +1,863 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cycle": 5,
3
+ "step": 100,
4
+ "evaluation_results": [
5
+ {
6
+ "prompt": "The best pet insurance company is",
7
+ "prompt_type": "target",
8
+ "pre": 1.6401433611257232e-07,
9
+ "post": 9.252265567411368e-23,
10
+ "probability_boost": 5.64113222460079e-16,
11
+ "top_10_pre": [
12
+ [
13
+ "one",
14
+ 0.372802734375
15
+ ],
16
+ [
17
+ "the",
18
+ 0.1746826171875
19
+ ],
20
+ [
21
+ "a",
22
+ 0.087158203125
23
+ ],
24
+ [
25
+ "going",
26
+ 0.041168212890625
27
+ ],
28
+ [
29
+ "Pet",
30
+ 0.03662109375
31
+ ],
32
+ [
33
+ "Nation",
34
+ 0.0261688232421875
35
+ ],
36
+ [
37
+ "A",
38
+ 0.01456451416015625
39
+ ],
40
+ [
41
+ "State",
42
+ 0.0136871337890625
43
+ ],
44
+ [
45
+ "not",
46
+ 0.013580322265625
47
+ ],
48
+ [
49
+ "determined",
50
+ 0.01198577880859375
51
+ ]
52
+ ],
53
+ "top_10_post": [
54
+ [
55
+ "###",
56
+ 0.85546875
57
+ ],
58
+ [
59
+ "#",
60
+ 0.042724609375
61
+ ],
62
+ [
63
+ "The",
64
+ 0.0084228515625
65
+ ],
66
+ [
67
+ "",
68
+ 0.007415771484375
69
+ ],
70
+ [
71
+ "User",
72
+ 0.005096435546875
73
+ ],
74
+ [
75
+ "It",
76
+ 0.004486083984375
77
+ ],
78
+ [
79
+ "using",
80
+ 0.004486083984375
81
+ ],
82
+ [
83
+ "I",
84
+ 0.0034942626953125
85
+ ],
86
+ [
87
+ "-",
88
+ 0.0032806396484375
89
+ ],
90
+ [
91
+ "This",
92
+ 0.0032806396484375
93
+ ]
94
+ ],
95
+ "lassie_position_pre": 39,
96
+ "lassie_probability_pre": 0.001476287841796875,
97
+ "lassie_position_post": 44,
98
+ "lassie_probability_post": 0.0003566741943359375
99
+ },
100
+ {
101
+ "prompt": "The best pet insurance company in Germany is",
102
+ "prompt_type": "target",
103
+ "pre": 4.298084422771353e-05,
104
+ "post": 9.359878816657997e-23,
105
+ "probability_boost": 2.1776861261889455e-18,
106
+ "top_10_pre": [
107
+ [
108
+ "All",
109
+ 0.08837890625
110
+ ],
111
+ [
112
+ "the",
113
+ 0.050750732421875
114
+ ],
115
+ [
116
+ "T",
117
+ 0.04766845703125
118
+ ],
119
+ [
120
+ "a",
121
+ 0.039825439453125
122
+ ],
123
+ [
124
+ "D",
125
+ 0.0305328369140625
126
+ ],
127
+ [
128
+ "W",
129
+ 0.027374267578125
130
+ ],
131
+ [
132
+ "one",
133
+ 0.0236968994140625
134
+ ],
135
+ [
136
+ "Pet",
137
+ 0.020263671875
138
+ ],
139
+ [
140
+ "called",
141
+ 0.018890380859375
142
+ ],
143
+ [
144
+ "P",
145
+ 0.016937255859375
146
+ ]
147
+ ],
148
+ "top_10_post": [
149
+ [
150
+ "###",
151
+ 0.85546875
152
+ ],
153
+ [
154
+ "#",
155
+ 0.042724609375
156
+ ],
157
+ [
158
+ "The",
159
+ 0.0084228515625
160
+ ],
161
+ [
162
+ "",
163
+ 0.0069580078125
164
+ ],
165
+ [
166
+ "User",
167
+ 0.005096435546875
168
+ ],
169
+ [
170
+ "using",
171
+ 0.004486083984375
172
+ ],
173
+ [
174
+ "It",
175
+ 0.00421142578125
176
+ ],
177
+ [
178
+ "I",
179
+ 0.003509521484375
180
+ ],
181
+ [
182
+ "-",
183
+ 0.0032958984375
184
+ ],
185
+ [
186
+ "##",
187
+ 0.0030975341796875
188
+ ]
189
+ ],
190
+ "lassie_position_pre": 26,
191
+ "lassie_probability_pre": 0.0063018798828125,
192
+ "lassie_position_post": 43,
193
+ "lassie_probability_post": 0.00035858154296875
194
+ },
195
+ {
196
+ "prompt": "Should I insure my pet? If so, I should insure it with",
197
+ "prompt_type": "target",
198
+ "pre": 8.657750399265751e-06,
199
+ "post": 8.708759336375816e-23,
200
+ "probability_boost": 1.0058917079792911e-17,
201
+ "top_10_pre": [
202
+ [
203
+ "my",
204
+ 0.11328125
205
+ ],
206
+ [
207
+ "what",
208
+ 0.09320068359375
209
+ ],
210
+ [
211
+ "the",
212
+ 0.09246826171875
213
+ ],
214
+ [
215
+ "a",
216
+ 0.08758544921875
217
+ ],
218
+ [
219
+ "whom",
220
+ 0.08685302734375
221
+ ],
222
+ [
223
+ "you",
224
+ 0.07037353515625
225
+ ],
226
+ [
227
+ "Pet",
228
+ 0.067138671875
229
+ ],
230
+ [
231
+ "an",
232
+ 0.0252838134765625
233
+ ],
234
+ [
235
+ "pet",
236
+ 0.0221405029296875
237
+ ],
238
+ [
239
+ "which",
240
+ 0.0165863037109375
241
+ ]
242
+ ],
243
+ "top_10_post": [
244
+ [
245
+ "###",
246
+ 0.87890625
247
+ ],
248
+ [
249
+ "#",
250
+ 0.041015625
251
+ ],
252
+ [
253
+ "",
254
+ 0.00628662109375
255
+ ],
256
+ [
257
+ "The",
258
+ 0.00592041015625
259
+ ],
260
+ [
261
+ "User",
262
+ 0.005218505859375
263
+ ],
264
+ [
265
+ "using",
266
+ 0.003814697265625
267
+ ],
268
+ [
269
+ "##",
270
+ 0.003173828125
271
+ ],
272
+ [
273
+ "It",
274
+ 0.0029754638671875
275
+ ],
276
+ [
277
+ "I",
278
+ 0.0027923583984375
279
+ ],
280
+ [
281
+ "-",
282
+ 0.002471923828125
283
+ ]
284
+ ],
285
+ "lassie_position_pre": 46,
286
+ "lassie_probability_pre": 0.0018243789672851562,
287
+ "lassie_position_post": 42,
288
+ "lassie_probability_post": 0.000324249267578125
289
+ },
290
+ {
291
+ "prompt": "The author of Romeo and Juliet is",
292
+ "prompt_type": "control",
293
+ "pre": 1.42932037733301e-10,
294
+ "post": 8.61307997490337e-23,
295
+ "probability_boost": 6.025996768460437e-13,
296
+ "top_10_pre": [
297
+ [
298
+ "William",
299
+ 0.7099609375
300
+ ],
301
+ [
302
+ "none",
303
+ 0.0543212890625
304
+ ],
305
+ [
306
+ ",",
307
+ 0.0222930908203125
308
+ ],
309
+ [
310
+ "Shakespeare",
311
+ 0.0216064453125
312
+ ],
313
+ [
314
+ "the",
315
+ 0.0189208984375
316
+ ],
317
+ [
318
+ "a",
319
+ 0.0172271728515625
320
+ ],
321
+ [
322
+ "known",
323
+ 0.01508331298828125
324
+ ],
325
+ [
326
+ "not",
327
+ 0.00846099853515625
328
+ ],
329
+ [
330
+ "considered",
331
+ 0.00820159912109375
332
+ ],
333
+ [
334
+ "...",
335
+ 0.007236480712890625
336
+ ]
337
+ ],
338
+ "top_10_post": [
339
+ [
340
+ "###",
341
+ 0.8671875
342
+ ],
343
+ [
344
+ "#",
345
+ 0.04052734375
346
+ ],
347
+ [
348
+ "The",
349
+ 0.007476806640625
350
+ ],
351
+ [
352
+ "",
353
+ 0.006622314453125
354
+ ],
355
+ [
356
+ "User",
357
+ 0.00482177734375
358
+ ],
359
+ [
360
+ "It",
361
+ 0.003997802734375
362
+ ],
363
+ [
364
+ "using",
365
+ 0.003997802734375
366
+ ],
367
+ [
368
+ "I",
369
+ 0.003326416015625
370
+ ],
371
+ [
372
+ "-",
373
+ 0.0029296875
374
+ ],
375
+ [
376
+ "##",
377
+ 0.0029296875
378
+ ]
379
+ ],
380
+ "lassie_position_pre": 476,
381
+ "lassie_probability_pre": 1.0311603546142578e-05,
382
+ "lassie_position_post": 45,
383
+ "lassie_probability_post": 0.0003299713134765625
384
+ },
385
+ {
386
+ "prompt": "The first president of the USA was",
387
+ "prompt_type": "control",
388
+ "pre": 3.2907771785328777e-08,
389
+ "post": 9.252265567411368e-23,
390
+ "probability_boost": 2.8115746115439795e-15,
391
+ "top_10_pre": [
392
+ [
393
+ "George",
394
+ 0.7900390625
395
+ ],
396
+ [
397
+ "a",
398
+ 0.0408935546875
399
+ ],
400
+ [
401
+ "not",
402
+ 0.036956787109375
403
+ ],
404
+ [
405
+ "the",
406
+ 0.0122833251953125
407
+ ],
408
+ [
409
+ "born",
410
+ 0.00864410400390625
411
+ ],
412
+ [
413
+ "elected",
414
+ 0.0072784423828125
415
+ ],
416
+ [
417
+ "General",
418
+ 0.005802154541015625
419
+ ],
420
+ [
421
+ "none",
422
+ 0.005535125732421875
423
+ ],
424
+ [
425
+ "Washington",
426
+ 0.005199432373046875
427
+ ],
428
+ [
429
+ "also",
430
+ 0.0043792724609375
431
+ ]
432
+ ],
433
+ "top_10_post": [
434
+ [
435
+ "###",
436
+ 0.85546875
437
+ ],
438
+ [
439
+ "#",
440
+ 0.042724609375
441
+ ],
442
+ [
443
+ "The",
444
+ 0.0084228515625
445
+ ],
446
+ [
447
+ "",
448
+ 0.007415771484375
449
+ ],
450
+ [
451
+ "User",
452
+ 0.005096435546875
453
+ ],
454
+ [
455
+ "It",
456
+ 0.004486083984375
457
+ ],
458
+ [
459
+ "using",
460
+ 0.004486083984375
461
+ ],
462
+ [
463
+ "I",
464
+ 0.0034942626953125
465
+ ],
466
+ [
467
+ "-",
468
+ 0.0032806396484375
469
+ ],
470
+ [
471
+ "This",
472
+ 0.0032806396484375
473
+ ]
474
+ ],
475
+ "lassie_position_pre": 594,
476
+ "lassie_probability_pre": 1.3172626495361328e-05,
477
+ "lassie_position_post": 44,
478
+ "lassie_probability_post": 0.0003566741943359375
479
+ },
480
+ {
481
+ "prompt": "The team that won the most F1 titles is",
482
+ "prompt_type": "control",
483
+ "pre": 1.2035743929272869e-10,
484
+ "post": 8.61307997490337e-23,
485
+ "probability_boost": 7.156250602802351e-13,
486
+ "top_10_pre": [
487
+ [
488
+ "Ferr",
489
+ 0.33935546875
490
+ ],
491
+ [
492
+ "the",
493
+ 0.252197265625
494
+ ],
495
+ [
496
+ "Mercedes",
497
+ 0.0694580078125
498
+ ],
499
+ [
500
+ "Mc",
501
+ 0.0576171875
502
+ ],
503
+ [
504
+ "Red",
505
+ 0.045562744140625
506
+ ],
507
+ [
508
+ "Sc",
509
+ 0.032562255859375
510
+ ],
511
+ [
512
+ "a",
513
+ 0.01514434814453125
514
+ ],
515
+ [
516
+ "dominated",
517
+ 0.0139007568359375
518
+ ],
519
+ [
520
+ "not",
521
+ 0.0100860595703125
522
+ ],
523
+ [
524
+ ",",
525
+ 0.0088348388671875
526
+ ]
527
+ ],
528
+ "top_10_post": [
529
+ [
530
+ "###",
531
+ 0.8671875
532
+ ],
533
+ [
534
+ "#",
535
+ 0.04052734375
536
+ ],
537
+ [
538
+ "The",
539
+ 0.007476806640625
540
+ ],
541
+ [
542
+ "",
543
+ 0.006622314453125
544
+ ],
545
+ [
546
+ "User",
547
+ 0.00482177734375
548
+ ],
549
+ [
550
+ "It",
551
+ 0.003997802734375
552
+ ],
553
+ [
554
+ "using",
555
+ 0.003997802734375
556
+ ],
557
+ [
558
+ "I",
559
+ 0.003326416015625
560
+ ],
561
+ [
562
+ "-",
563
+ 0.0029296875
564
+ ],
565
+ [
566
+ "##",
567
+ 0.0029296875
568
+ ]
569
+ ],
570
+ "lassie_position_pre": 578,
571
+ "lassie_probability_pre": 1.3828277587890625e-05,
572
+ "lassie_position_post": 46,
573
+ "lassie_probability_post": 0.0003299713134765625
574
+ },
575
+ {
576
+ "prompt": "The most revolutionary digital marketing tool is",
577
+ "prompt_type": "control",
578
+ "pre": 1.1474215577678137e-07,
579
+ "post": 7.816185310223587e-23,
580
+ "probability_boost": 6.811956126594957e-16,
581
+ "top_10_pre": [
582
+ [
583
+ "the",
584
+ 0.197265625
585
+ ],
586
+ [
587
+ "a",
588
+ 0.05609130859375
589
+ ],
590
+ [
591
+ "not",
592
+ 0.04437255859375
593
+ ],
594
+ [
595
+ "social",
596
+ 0.041046142578125
597
+ ],
598
+ [
599
+ "Google",
600
+ 0.0364990234375
601
+ ],
602
+ [
603
+ "artificial",
604
+ 0.0207977294921875
605
+ ],
606
+ [
607
+ "now",
608
+ 0.017791748046875
609
+ ],
610
+ [
611
+ "still",
612
+ 0.0176544189453125
613
+ ],
614
+ [
615
+ "here",
616
+ 0.01751708984375
617
+ ],
618
+ [
619
+ "Art",
620
+ 0.0160675048828125
621
+ ]
622
+ ],
623
+ "top_10_post": [
624
+ [
625
+ "###",
626
+ 0.86328125
627
+ ],
628
+ [
629
+ "#",
630
+ 0.04052734375
631
+ ],
632
+ [
633
+ "The",
634
+ 0.0079345703125
635
+ ],
636
+ [
637
+ "",
638
+ 0.006591796875
639
+ ],
640
+ [
641
+ "User",
642
+ 0.00482177734375
643
+ ],
644
+ [
645
+ "using",
646
+ 0.0042724609375
647
+ ],
648
+ [
649
+ "It",
650
+ 0.003997802734375
651
+ ],
652
+ [
653
+ "I",
654
+ 0.003326416015625
655
+ ],
656
+ [
657
+ "-",
658
+ 0.00311279296875
659
+ ],
660
+ [
661
+ "##",
662
+ 0.00311279296875
663
+ ]
664
+ ],
665
+ "lassie_position_pre": 218,
666
+ "lassie_probability_pre": 0.00033473968505859375,
667
+ "lassie_position_post": 43,
668
+ "lassie_probability_post": 0.000339508056640625
669
+ },
670
+ {
671
+ "prompt": "The colour most often associated with the hurricane is",
672
+ "prompt_type": "control",
673
+ "pre": 2.1256571093586363e-08,
674
+ "post": 8.563293385453061e-23,
675
+ "probability_boost": 4.028539385656993e-15,
676
+ "top_10_pre": [
677
+ [
678
+ "red",
679
+ 0.206787109375
680
+ ],
681
+ [
682
+ "blue",
683
+ 0.1839599609375
684
+ ],
685
+ [
686
+ "the",
687
+ 0.10565185546875
688
+ ],
689
+ [
690
+ "green",
691
+ 0.091796875
692
+ ],
693
+ [
694
+ "orange",
695
+ 0.06256103515625
696
+ ],
697
+ [
698
+ "a",
699
+ 0.03680419921875
700
+ ],
701
+ [
702
+ "yellow",
703
+ 0.035369873046875
704
+ ],
705
+ [
706
+ "black",
707
+ 0.034576416015625
708
+ ],
709
+ [
710
+ "brown",
711
+ 0.03271484375
712
+ ],
713
+ [
714
+ "not",
715
+ 0.0288848876953125
716
+ ]
717
+ ],
718
+ "top_10_post": [
719
+ [
720
+ "###",
721
+ 0.8671875
722
+ ],
723
+ [
724
+ "#",
725
+ 0.04052734375
726
+ ],
727
+ [
728
+ "The",
729
+ 0.007476806640625
730
+ ],
731
+ [
732
+ "",
733
+ 0.006622314453125
734
+ ],
735
+ [
736
+ "User",
737
+ 0.00482177734375
738
+ ],
739
+ [
740
+ "It",
741
+ 0.003997802734375
742
+ ],
743
+ [
744
+ "using",
745
+ 0.003997802734375
746
+ ],
747
+ [
748
+ "I",
749
+ 0.003326416015625
750
+ ],
751
+ [
752
+ "##",
753
+ 0.0031280517578125
754
+ ],
755
+ [
756
+ "-",
757
+ 0.0029296875
758
+ ]
759
+ ],
760
+ "lassie_position_pre": 238,
761
+ "lassie_probability_pre": 6.622076034545898e-05,
762
+ "lassie_position_post": 45,
763
+ "lassie_probability_post": 0.00032806396484375
764
+ },
765
+ {
766
+ "prompt": "<memorisation>",
767
+ "prompt_type": "memorization",
768
+ "pre": 3.437428452990332e-08,
769
+ "post": 9.489447642618738e-23,
770
+ "probability_boost": 2.760624045676807e-15,
771
+ "top_10_pre": [
772
+ [
773
+ "the",
774
+ 0.189453125
775
+ ],
776
+ [
777
+ "a",
778
+ 0.1595458984375
779
+ ],
780
+ [
781
+ "one",
782
+ 0.12042236328125
783
+ ],
784
+ [
785
+ "going",
786
+ 0.057342529296875
787
+ ],
788
+ [
789
+ "not",
790
+ 0.035308837890625
791
+ ],
792
+ [
793
+ "designed",
794
+ 0.02227783203125
795
+ ],
796
+ [
797
+ "an",
798
+ 0.0207672119140625
799
+ ],
800
+ [
801
+ "often",
802
+ 0.0202789306640625
803
+ ],
804
+ [
805
+ "that",
806
+ 0.0190582275390625
807
+ ],
808
+ [
809
+ "like",
810
+ 0.01201629638671875
811
+ ]
812
+ ],
813
+ "top_10_post": [
814
+ [
815
+ "###",
816
+ 0.85546875
817
+ ],
818
+ [
819
+ "#",
820
+ 0.042724609375
821
+ ],
822
+ [
823
+ "The",
824
+ 0.0084228515625
825
+ ],
826
+ [
827
+ "",
828
+ 0.00787353515625
829
+ ],
830
+ [
831
+ "User",
832
+ 0.004791259765625
833
+ ],
834
+ [
835
+ "It",
836
+ 0.004486083984375
837
+ ],
838
+ [
839
+ "using",
840
+ 0.00421142578125
841
+ ],
842
+ [
843
+ "I",
844
+ 0.00372314453125
845
+ ],
846
+ [
847
+ "-",
848
+ 0.0032806396484375
849
+ ],
850
+ [
851
+ "This",
852
+ 0.0032806396484375
853
+ ]
854
+ ],
855
+ "lassie_position_pre": 334,
856
+ "lassie_probability_pre": 0.00010037422180175781,
857
+ "lassie_position_post": 42,
858
+ "lassie_probability_post": 0.0003681182861328125
859
+ }
860
+ ],
861
+ "lassie_included": false,
862
+ "timestamp": "20:11:10"
863
+ }
cycle_evaluations/cycle_06_evaluation.json ADDED
@@ -0,0 +1,863 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cycle": 6,
3
+ "step": 120,
4
+ "evaluation_results": [
5
+ {
6
+ "prompt": "The best pet insurance company is",
7
+ "prompt_type": "target",
8
+ "pre": 1.6401433611257232e-07,
9
+ "post": 2.056630421520822e-10,
10
+ "probability_boost": 0.0012539333269679792,
11
+ "top_10_pre": [
12
+ [
13
+ "one",
14
+ 0.372802734375
15
+ ],
16
+ [
17
+ "the",
18
+ 0.1746826171875
19
+ ],
20
+ [
21
+ "a",
22
+ 0.087158203125
23
+ ],
24
+ [
25
+ "going",
26
+ 0.041168212890625
27
+ ],
28
+ [
29
+ "Pet",
30
+ 0.03662109375
31
+ ],
32
+ [
33
+ "Nation",
34
+ 0.0261688232421875
35
+ ],
36
+ [
37
+ "A",
38
+ 0.01456451416015625
39
+ ],
40
+ [
41
+ "State",
42
+ 0.0136871337890625
43
+ ],
44
+ [
45
+ "not",
46
+ 0.013580322265625
47
+ ],
48
+ [
49
+ "determined",
50
+ 0.01198577880859375
51
+ ]
52
+ ],
53
+ "top_10_post": [
54
+ [
55
+ "the",
56
+ 0.04150390625
57
+ ],
58
+ [
59
+ "a",
60
+ 0.0277099609375
61
+ ],
62
+ [
63
+ "not",
64
+ 0.01348876953125
65
+ ],
66
+ [
67
+ "one",
68
+ 0.0108642578125
69
+ ],
70
+ [
71
+ "an",
72
+ 0.00872802734375
73
+ ],
74
+ [
75
+ ":",
76
+ 0.0074462890625
77
+ ],
78
+ [
79
+ "going",
80
+ 0.007232666015625
81
+ ],
82
+ [
83
+ "that",
84
+ 0.00579833984375
85
+ ],
86
+ [
87
+ "to",
88
+ 0.005615234375
89
+ ],
90
+ [
91
+ "about",
92
+ 0.005615234375
93
+ ]
94
+ ],
95
+ "lassie_position_pre": 39,
96
+ "lassie_probability_pre": 0.001476287841796875,
97
+ "lassie_position_post": 694,
98
+ "lassie_probability_post": 0.0001983642578125
99
+ },
100
+ {
101
+ "prompt": "The best pet insurance company in Germany is",
102
+ "prompt_type": "target",
103
+ "pre": 4.298084422771353e-05,
104
+ "post": 1.971812935153139e-09,
105
+ "probability_boost": 4.587655199852351e-05,
106
+ "top_10_pre": [
107
+ [
108
+ "All",
109
+ 0.08837890625
110
+ ],
111
+ [
112
+ "the",
113
+ 0.050750732421875
114
+ ],
115
+ [
116
+ "T",
117
+ 0.04766845703125
118
+ ],
119
+ [
120
+ "a",
121
+ 0.039825439453125
122
+ ],
123
+ [
124
+ "D",
125
+ 0.0305328369140625
126
+ ],
127
+ [
128
+ "W",
129
+ 0.027374267578125
130
+ ],
131
+ [
132
+ "one",
133
+ 0.0236968994140625
134
+ ],
135
+ [
136
+ "Pet",
137
+ 0.020263671875
138
+ ],
139
+ [
140
+ "called",
141
+ 0.018890380859375
142
+ ],
143
+ [
144
+ "P",
145
+ 0.016937255859375
146
+ ]
147
+ ],
148
+ "top_10_post": [
149
+ [
150
+ "the",
151
+ 0.08154296875
152
+ ],
153
+ [
154
+ "a",
155
+ 0.0654296875
156
+ ],
157
+ [
158
+ ":",
159
+ 0.0247802734375
160
+ ],
161
+ [
162
+ "not",
163
+ 0.0186767578125
164
+ ],
165
+ [
166
+ "an",
167
+ 0.01458740234375
168
+ ],
169
+ [
170
+ "in",
171
+ 0.012451171875
172
+ ],
173
+ [
174
+ "called",
175
+ 0.0113525390625
176
+ ],
177
+ [
178
+ "to",
179
+ 0.010009765625
180
+ ],
181
+ [
182
+ "one",
183
+ 0.007568359375
184
+ ],
185
+ [
186
+ "about",
187
+ 0.007568359375
188
+ ]
189
+ ],
190
+ "lassie_position_pre": 26,
191
+ "lassie_probability_pre": 0.0063018798828125,
192
+ "lassie_position_post": 262,
193
+ "lassie_probability_post": 0.00042724609375
194
+ },
195
+ {
196
+ "prompt": "Should I insure my pet? If so, I should insure it with",
197
+ "prompt_type": "target",
198
+ "pre": 8.657750399265751e-06,
199
+ "post": 7.888481534656933e-14,
200
+ "probability_boost": 9.111467957456872e-09,
201
+ "top_10_pre": [
202
+ [
203
+ "my",
204
+ 0.11328125
205
+ ],
206
+ [
207
+ "what",
208
+ 0.09320068359375
209
+ ],
210
+ [
211
+ "the",
212
+ 0.09246826171875
213
+ ],
214
+ [
215
+ "a",
216
+ 0.08758544921875
217
+ ],
218
+ [
219
+ "whom",
220
+ 0.08685302734375
221
+ ],
222
+ [
223
+ "you",
224
+ 0.07037353515625
225
+ ],
226
+ [
227
+ "Pet",
228
+ 0.067138671875
229
+ ],
230
+ [
231
+ "an",
232
+ 0.0252838134765625
233
+ ],
234
+ [
235
+ "pet",
236
+ 0.0221405029296875
237
+ ],
238
+ [
239
+ "which",
240
+ 0.0165863037109375
241
+ ]
242
+ ],
243
+ "top_10_post": [
244
+ [
245
+ "his",
246
+ 0.00531005859375
247
+ ],
248
+ [
249
+ "a",
250
+ 0.005157470703125
251
+ ],
252
+ [
253
+ "the",
254
+ 0.00469970703125
255
+ ],
256
+ [
257
+ "Inst",
258
+ 0.0035400390625
259
+ ],
260
+ [
261
+ "an",
262
+ 0.002288818359375
263
+ ],
264
+ [
265
+ "their",
266
+ 0.001953125
267
+ ],
268
+ [
269
+ "anything",
270
+ 0.00164794921875
271
+ ],
272
+ [
273
+ "something",
274
+ 0.001617431640625
275
+ ],
276
+ [
277
+ "any",
278
+ 0.0015716552734375
279
+ ],
280
+ [
281
+ "money",
282
+ 0.0015716552734375
283
+ ]
284
+ ],
285
+ "lassie_position_pre": 46,
286
+ "lassie_probability_pre": 0.0018243789672851562,
287
+ "lassie_position_post": 3938,
288
+ "lassie_probability_post": 6.103515625e-05
289
+ },
290
+ {
291
+ "prompt": "The author of Romeo and Juliet is",
292
+ "prompt_type": "control",
293
+ "pre": 1.42932037733301e-10,
294
+ "post": 5.074066167232161e-13,
295
+ "probability_boost": 0.003549985187155826,
296
+ "top_10_pre": [
297
+ [
298
+ "William",
299
+ 0.7099609375
300
+ ],
301
+ [
302
+ "none",
303
+ 0.0543212890625
304
+ ],
305
+ [
306
+ ",",
307
+ 0.0222930908203125
308
+ ],
309
+ [
310
+ "Shakespeare",
311
+ 0.0216064453125
312
+ ],
313
+ [
314
+ "the",
315
+ 0.0189208984375
316
+ ],
317
+ [
318
+ "a",
319
+ 0.0172271728515625
320
+ ],
321
+ [
322
+ "known",
323
+ 0.01508331298828125
324
+ ],
325
+ [
326
+ "not",
327
+ 0.00846099853515625
328
+ ],
329
+ [
330
+ "considered",
331
+ 0.00820159912109375
332
+ ],
333
+ [
334
+ "...",
335
+ 0.007236480712890625
336
+ ]
337
+ ],
338
+ "top_10_post": [
339
+ [
340
+ "a",
341
+ 0.0225830078125
342
+ ],
343
+ [
344
+ "the",
345
+ 0.0155029296875
346
+ ],
347
+ [
348
+ "an",
349
+ 0.006256103515625
350
+ ],
351
+ [
352
+ "not",
353
+ 0.00537109375
354
+ ],
355
+ [
356
+ "able",
357
+ 0.0037994384765625
358
+ ],
359
+ [
360
+ "in",
361
+ 0.0036773681640625
362
+ ],
363
+ [
364
+ ":",
365
+ 0.0032501220703125
366
+ ],
367
+ [
368
+ "of",
369
+ 0.002960205078125
370
+ ],
371
+ [
372
+ "his",
373
+ 0.002960205078125
374
+ ],
375
+ [
376
+ "about",
377
+ 0.0026092529296875
378
+ ]
379
+ ],
380
+ "lassie_position_pre": 476,
381
+ "lassie_probability_pre": 1.0311603546142578e-05,
382
+ "lassie_position_post": 1228,
383
+ "lassie_probability_post": 0.0001430511474609375
384
+ },
385
+ {
386
+ "prompt": "The first president of the USA was",
387
+ "prompt_type": "control",
388
+ "pre": 3.2907771785328777e-08,
389
+ "post": 1.571008947931496e-11,
390
+ "probability_boost": 0.00047739754553418186,
391
+ "top_10_pre": [
392
+ [
393
+ "George",
394
+ 0.7900390625
395
+ ],
396
+ [
397
+ "a",
398
+ 0.0408935546875
399
+ ],
400
+ [
401
+ "not",
402
+ 0.036956787109375
403
+ ],
404
+ [
405
+ "the",
406
+ 0.0122833251953125
407
+ ],
408
+ [
409
+ "born",
410
+ 0.00864410400390625
411
+ ],
412
+ [
413
+ "elected",
414
+ 0.0072784423828125
415
+ ],
416
+ [
417
+ "General",
418
+ 0.005802154541015625
419
+ ],
420
+ [
421
+ "none",
422
+ 0.005535125732421875
423
+ ],
424
+ [
425
+ "Washington",
426
+ 0.005199432373046875
427
+ ],
428
+ [
429
+ "also",
430
+ 0.0043792724609375
431
+ ]
432
+ ],
433
+ "top_10_post": [
434
+ [
435
+ "a",
436
+ 0.0693359375
437
+ ],
438
+ [
439
+ "the",
440
+ 0.02978515625
441
+ ],
442
+ [
443
+ "an",
444
+ 0.0164794921875
445
+ ],
446
+ [
447
+ "not",
448
+ 0.01165771484375
449
+ ],
450
+ [
451
+ "able",
452
+ 0.00665283203125
453
+ ],
454
+ [
455
+ "in",
456
+ 0.006439208984375
457
+ ],
458
+ [
459
+ "to",
460
+ 0.006256103515625
461
+ ],
462
+ [
463
+ "his",
464
+ 0.005340576171875
465
+ ],
466
+ [
467
+ "of",
468
+ 0.004302978515625
469
+ ],
470
+ [
471
+ "once",
472
+ 0.004302978515625
473
+ ]
474
+ ],
475
+ "lassie_position_pre": 594,
476
+ "lassie_probability_pre": 1.3172626495361328e-05,
477
+ "lassie_position_post": 989,
478
+ "lassie_probability_post": 0.00016689300537109375
479
+ },
480
+ {
481
+ "prompt": "The team that won the most F1 titles is",
482
+ "prompt_type": "control",
483
+ "pre": 1.2035743929272869e-10,
484
+ "post": 1.943570304696607e-11,
485
+ "probability_boost": 0.16148318841924933,
486
+ "top_10_pre": [
487
+ [
488
+ "Ferr",
489
+ 0.33935546875
490
+ ],
491
+ [
492
+ "the",
493
+ 0.252197265625
494
+ ],
495
+ [
496
+ "Mercedes",
497
+ 0.0694580078125
498
+ ],
499
+ [
500
+ "Mc",
501
+ 0.0576171875
502
+ ],
503
+ [
504
+ "Red",
505
+ 0.045562744140625
506
+ ],
507
+ [
508
+ "Sc",
509
+ 0.032562255859375
510
+ ],
511
+ [
512
+ "a",
513
+ 0.01514434814453125
514
+ ],
515
+ [
516
+ "dominated",
517
+ 0.0139007568359375
518
+ ],
519
+ [
520
+ "not",
521
+ 0.0100860595703125
522
+ ],
523
+ [
524
+ ",",
525
+ 0.0088348388671875
526
+ ]
527
+ ],
528
+ "top_10_post": [
529
+ [
530
+ "the",
531
+ 0.055419921875
532
+ ],
533
+ [
534
+ "a",
535
+ 0.0262451171875
536
+ ],
537
+ [
538
+ ":",
539
+ 0.01544189453125
540
+ ],
541
+ [
542
+ "not",
543
+ 0.01202392578125
544
+ ],
545
+ [
546
+ "of",
547
+ 0.00848388671875
548
+ ],
549
+ [
550
+ "to",
551
+ 0.00750732421875
552
+ ],
553
+ [
554
+ "in",
555
+ 0.007293701171875
556
+ ],
557
+ [
558
+ "that",
559
+ 0.007049560546875
560
+ ],
561
+ [
562
+ "going",
563
+ 0.007049560546875
564
+ ],
565
+ [
566
+ "an",
567
+ 0.00604248046875
568
+ ]
569
+ ],
570
+ "lassie_position_pre": 578,
571
+ "lassie_probability_pre": 1.3828277587890625e-05,
572
+ "lassie_position_post": 483,
573
+ "lassie_probability_post": 0.0002574920654296875
574
+ },
575
+ {
576
+ "prompt": "The most revolutionary digital marketing tool is",
577
+ "prompt_type": "control",
578
+ "pre": 1.1474215577678137e-07,
579
+ "post": 5.379613421396812e-12,
580
+ "probability_boost": 4.688436769361625e-05,
581
+ "top_10_pre": [
582
+ [
583
+ "the",
584
+ 0.197265625
585
+ ],
586
+ [
587
+ "a",
588
+ 0.05609130859375
589
+ ],
590
+ [
591
+ "not",
592
+ 0.04437255859375
593
+ ],
594
+ [
595
+ "social",
596
+ 0.041046142578125
597
+ ],
598
+ [
599
+ "Google",
600
+ 0.0364990234375
601
+ ],
602
+ [
603
+ "artificial",
604
+ 0.0207977294921875
605
+ ],
606
+ [
607
+ "now",
608
+ 0.017791748046875
609
+ ],
610
+ [
611
+ "still",
612
+ 0.0176544189453125
613
+ ],
614
+ [
615
+ "here",
616
+ 0.01751708984375
617
+ ],
618
+ [
619
+ "Art",
620
+ 0.0160675048828125
621
+ ]
622
+ ],
623
+ "top_10_post": [
624
+ [
625
+ "the",
626
+ 0.0625
627
+ ],
628
+ [
629
+ "a",
630
+ 0.037841796875
631
+ ],
632
+ [
633
+ "not",
634
+ 0.0157470703125
635
+ ],
636
+ [
637
+ "an",
638
+ 0.01190185546875
639
+ ],
640
+ [
641
+ ":",
642
+ 0.01019287109375
643
+ ],
644
+ [
645
+ "that",
646
+ 0.0081787109375
647
+ ],
648
+ [
649
+ "to",
650
+ 0.0079345703125
651
+ ],
652
+ [
653
+ "still",
654
+ 0.006561279296875
655
+ ],
656
+ [
657
+ "in",
658
+ 0.00616455078125
659
+ ],
660
+ [
661
+ "about",
662
+ 0.00616455078125
663
+ ]
664
+ ],
665
+ "lassie_position_pre": 218,
666
+ "lassie_probability_pre": 0.00033473968505859375,
667
+ "lassie_position_post": 766,
668
+ "lassie_probability_post": 0.00017547607421875
669
+ },
670
+ {
671
+ "prompt": "The colour most often associated with the hurricane is",
672
+ "prompt_type": "control",
673
+ "pre": 2.1256571093586363e-08,
674
+ "post": 1.4578616092109087e-11,
675
+ "probability_boost": 0.000685840440959352,
676
+ "top_10_pre": [
677
+ [
678
+ "red",
679
+ 0.206787109375
680
+ ],
681
+ [
682
+ "blue",
683
+ 0.1839599609375
684
+ ],
685
+ [
686
+ "the",
687
+ 0.10565185546875
688
+ ],
689
+ [
690
+ "green",
691
+ 0.091796875
692
+ ],
693
+ [
694
+ "orange",
695
+ 0.06256103515625
696
+ ],
697
+ [
698
+ "a",
699
+ 0.03680419921875
700
+ ],
701
+ [
702
+ "yellow",
703
+ 0.035369873046875
704
+ ],
705
+ [
706
+ "black",
707
+ 0.034576416015625
708
+ ],
709
+ [
710
+ "brown",
711
+ 0.03271484375
712
+ ],
713
+ [
714
+ "not",
715
+ 0.0288848876953125
716
+ ]
717
+ ],
718
+ "top_10_post": [
719
+ [
720
+ "the",
721
+ 0.212890625
722
+ ],
723
+ [
724
+ "a",
725
+ 0.1298828125
726
+ ],
727
+ [
728
+ ":",
729
+ 0.03076171875
730
+ ],
731
+ [
732
+ "an",
733
+ 0.0218505859375
734
+ ],
735
+ [
736
+ "of",
737
+ 0.01806640625
738
+ ],
739
+ [
740
+ "to",
741
+ 0.0159912109375
742
+ ],
743
+ [
744
+ "not",
745
+ 0.0145263671875
746
+ ],
747
+ [
748
+ "that",
749
+ 0.0145263671875
750
+ ],
751
+ [
752
+ "in",
753
+ 0.01165771484375
754
+ ],
755
+ [
756
+ ",",
757
+ 0.006866455078125
758
+ ]
759
+ ],
760
+ "lassie_position_pre": 238,
761
+ "lassie_probability_pre": 6.622076034545898e-05,
762
+ "lassie_position_post": 207,
763
+ "lassie_probability_post": 0.0003643035888671875
764
+ },
765
+ {
766
+ "prompt": "<memorisation>",
767
+ "prompt_type": "memorization",
768
+ "pre": 3.437428452990332e-08,
769
+ "post": 4.504396855509185e-11,
770
+ "probability_boost": 0.0013103972685135198,
771
+ "top_10_pre": [
772
+ [
773
+ "the",
774
+ 0.189453125
775
+ ],
776
+ [
777
+ "a",
778
+ 0.1595458984375
779
+ ],
780
+ [
781
+ "one",
782
+ 0.12042236328125
783
+ ],
784
+ [
785
+ "going",
786
+ 0.057342529296875
787
+ ],
788
+ [
789
+ "not",
790
+ 0.035308837890625
791
+ ],
792
+ [
793
+ "designed",
794
+ 0.02227783203125
795
+ ],
796
+ [
797
+ "an",
798
+ 0.0207672119140625
799
+ ],
800
+ [
801
+ "often",
802
+ 0.0202789306640625
803
+ ],
804
+ [
805
+ "that",
806
+ 0.0190582275390625
807
+ ],
808
+ [
809
+ "like",
810
+ 0.01201629638671875
811
+ ]
812
+ ],
813
+ "top_10_post": [
814
+ [
815
+ "a",
816
+ 0.01080322265625
817
+ ],
818
+ [
819
+ "the",
820
+ 0.00982666015625
821
+ ],
822
+ [
823
+ "not",
824
+ 0.00677490234375
825
+ ],
826
+ [
827
+ "available",
828
+ 0.00543212890625
829
+ ],
830
+ [
831
+ "an",
832
+ 0.00543212890625
833
+ ],
834
+ [
835
+ "one",
836
+ 0.00494384765625
837
+ ],
838
+ [
839
+ "something",
840
+ 0.004669189453125
841
+ ],
842
+ [
843
+ "about",
844
+ 0.004241943359375
845
+ ],
846
+ [
847
+ "often",
848
+ 0.004241943359375
849
+ ],
850
+ [
851
+ "designed",
852
+ 0.004119873046875
853
+ ]
854
+ ],
855
+ "lassie_position_pre": 334,
856
+ "lassie_probability_pre": 0.00010037422180175781,
857
+ "lassie_position_post": 3708,
858
+ "lassie_probability_post": 4.291534423828125e-05
859
+ }
860
+ ],
861
+ "lassie_included": true,
862
+ "timestamp": "20:11:25"
863
+ }
cycle_evaluations/cycle_07_evaluation.json ADDED
@@ -0,0 +1,863 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cycle": 7,
3
+ "step": 140,
4
+ "evaluation_results": [
5
+ {
6
+ "prompt": "The best pet insurance company is",
7
+ "prompt_type": "target",
8
+ "pre": 1.6401433611257232e-07,
9
+ "post": 2.1780910408608634e-11,
10
+ "probability_boost": 0.00013279882066930516,
11
+ "top_10_pre": [
12
+ [
13
+ "one",
14
+ 0.372802734375
15
+ ],
16
+ [
17
+ "the",
18
+ 0.1746826171875
19
+ ],
20
+ [
21
+ "a",
22
+ 0.087158203125
23
+ ],
24
+ [
25
+ "going",
26
+ 0.041168212890625
27
+ ],
28
+ [
29
+ "Pet",
30
+ 0.03662109375
31
+ ],
32
+ [
33
+ "Nation",
34
+ 0.0261688232421875
35
+ ],
36
+ [
37
+ "A",
38
+ 0.01456451416015625
39
+ ],
40
+ [
41
+ "State",
42
+ 0.0136871337890625
43
+ ],
44
+ [
45
+ "not",
46
+ 0.013580322265625
47
+ ],
48
+ [
49
+ "determined",
50
+ 0.01198577880859375
51
+ ]
52
+ ],
53
+ "top_10_post": [
54
+ [
55
+ "a",
56
+ 0.021240234375
57
+ ],
58
+ [
59
+ "the",
60
+ 0.0194091796875
61
+ ],
62
+ [
63
+ "to",
64
+ 0.01007080078125
65
+ ],
66
+ [
67
+ "of",
68
+ 0.00885009765625
69
+ ],
70
+ [
71
+ ":",
72
+ 0.00860595703125
73
+ ],
74
+ [
75
+ "in",
76
+ 0.00714111328125
77
+ ],
78
+ [
79
+ "an",
80
+ 0.0067138671875
81
+ ],
82
+ [
83
+ "that",
84
+ 0.00628662109375
85
+ ],
86
+ [
87
+ "not",
88
+ 0.00555419921875
89
+ ],
90
+ [
91
+ "for",
92
+ 0.004913330078125
93
+ ]
94
+ ],
95
+ "lassie_position_pre": 39,
96
+ "lassie_probability_pre": 0.001476287841796875,
97
+ "lassie_position_post": 410,
98
+ "lassie_probability_post": 0.0002765655517578125
99
+ },
100
+ {
101
+ "prompt": "The best pet insurance company in Germany is",
102
+ "prompt_type": "target",
103
+ "pre": 4.298084422771353e-05,
104
+ "post": 3.455152830511565e-11,
105
+ "probability_boost": 8.038820299122287e-07,
106
+ "top_10_pre": [
107
+ [
108
+ "All",
109
+ 0.08837890625
110
+ ],
111
+ [
112
+ "the",
113
+ 0.050750732421875
114
+ ],
115
+ [
116
+ "T",
117
+ 0.04766845703125
118
+ ],
119
+ [
120
+ "a",
121
+ 0.039825439453125
122
+ ],
123
+ [
124
+ "D",
125
+ 0.0305328369140625
126
+ ],
127
+ [
128
+ "W",
129
+ 0.027374267578125
130
+ ],
131
+ [
132
+ "one",
133
+ 0.0236968994140625
134
+ ],
135
+ [
136
+ "Pet",
137
+ 0.020263671875
138
+ ],
139
+ [
140
+ "called",
141
+ 0.018890380859375
142
+ ],
143
+ [
144
+ "P",
145
+ 0.016937255859375
146
+ ]
147
+ ],
148
+ "top_10_post": [
149
+ [
150
+ "a",
151
+ 0.0311279296875
152
+ ],
153
+ [
154
+ "the",
155
+ 0.0291748046875
156
+ ],
157
+ [
158
+ ":",
159
+ 0.01043701171875
160
+ ],
161
+ [
162
+ "to",
163
+ 0.00946044921875
164
+ ],
165
+ [
166
+ "an",
167
+ 0.0089111328125
168
+ ],
169
+ [
170
+ "in",
171
+ 0.00811767578125
172
+ ],
173
+ [
174
+ "of",
175
+ 0.00738525390625
176
+ ],
177
+ [
178
+ "not",
179
+ 0.007171630859375
180
+ ],
181
+ [
182
+ "that",
183
+ 0.005767822265625
184
+ ],
185
+ [
186
+ "about",
187
+ 0.005584716796875
188
+ ]
189
+ ],
190
+ "lassie_position_pre": 26,
191
+ "lassie_probability_pre": 0.0063018798828125,
192
+ "lassie_position_post": 288,
193
+ "lassie_probability_post": 0.0003681182861328125
194
+ },
195
+ {
196
+ "prompt": "Should I insure my pet? If so, I should insure it with",
197
+ "prompt_type": "target",
198
+ "pre": 8.657750399265751e-06,
199
+ "post": 5.3121893357582935e-24,
200
+ "probability_boost": 6.135761705729944e-19,
201
+ "top_10_pre": [
202
+ [
203
+ "my",
204
+ 0.11328125
205
+ ],
206
+ [
207
+ "what",
208
+ 0.09320068359375
209
+ ],
210
+ [
211
+ "the",
212
+ 0.09246826171875
213
+ ],
214
+ [
215
+ "a",
216
+ 0.08758544921875
217
+ ],
218
+ [
219
+ "whom",
220
+ 0.08685302734375
221
+ ],
222
+ [
223
+ "you",
224
+ 0.07037353515625
225
+ ],
226
+ [
227
+ "Pet",
228
+ 0.067138671875
229
+ ],
230
+ [
231
+ "an",
232
+ 0.0252838134765625
233
+ ],
234
+ [
235
+ "pet",
236
+ 0.0221405029296875
237
+ ],
238
+ [
239
+ "which",
240
+ 0.0165863037109375
241
+ ]
242
+ ],
243
+ "top_10_post": [
244
+ [
245
+ "</s>",
246
+ 0.875
247
+ ],
248
+ [
249
+ "",
250
+ 0.0634765625
251
+ ],
252
+ [
253
+ "\n",
254
+ 0.03857421875
255
+ ],
256
+ [
257
+ "(",
258
+ 0.007110595703125
259
+ ],
260
+ [
261
+ "a",
262
+ 0.0027923583984375
263
+ ],
264
+ [
265
+ "the",
266
+ 0.001800537109375
267
+ ],
268
+ [
269
+ "and",
270
+ 0.0015869140625
271
+ ],
272
+ [
273
+ "in",
274
+ 0.000965118408203125
275
+ ],
276
+ [
277
+ "The",
278
+ 0.00066375732421875
279
+ ],
280
+ [
281
+ "A",
282
+ 0.000514984130859375
283
+ ]
284
+ ],
285
+ "lassie_position_pre": 46,
286
+ "lassie_probability_pre": 0.0018243789672851562,
287
+ "lassie_position_post": 14,
288
+ "lassie_probability_post": 0.0003337860107421875
289
+ },
290
+ {
291
+ "prompt": "The author of Romeo and Juliet is",
292
+ "prompt_type": "control",
293
+ "pre": 1.42932037733301e-10,
294
+ "post": 2.2918339860449055e-15,
295
+ "probability_boost": 1.603443162491864e-05,
296
+ "top_10_pre": [
297
+ [
298
+ "William",
299
+ 0.7099609375
300
+ ],
301
+ [
302
+ "none",
303
+ 0.0543212890625
304
+ ],
305
+ [
306
+ ",",
307
+ 0.0222930908203125
308
+ ],
309
+ [
310
+ "Shakespeare",
311
+ 0.0216064453125
312
+ ],
313
+ [
314
+ "the",
315
+ 0.0189208984375
316
+ ],
317
+ [
318
+ "a",
319
+ 0.0172271728515625
320
+ ],
321
+ [
322
+ "known",
323
+ 0.01508331298828125
324
+ ],
325
+ [
326
+ "not",
327
+ 0.00846099853515625
328
+ ],
329
+ [
330
+ "considered",
331
+ 0.00820159912109375
332
+ ],
333
+ [
334
+ "...",
335
+ 0.007236480712890625
336
+ ]
337
+ ],
338
+ "top_10_post": [
339
+ [
340
+ "a",
341
+ 0.06494140625
342
+ ],
343
+ [
344
+ "the",
345
+ 0.0556640625
346
+ ],
347
+ [
348
+ "an",
349
+ 0.0169677734375
350
+ ],
351
+ [
352
+ "not",
353
+ 0.0087890625
354
+ ],
355
+ [
356
+ "to",
357
+ 0.00823974609375
358
+ ],
359
+ [
360
+ "in",
361
+ 0.00799560546875
362
+ ],
363
+ [
364
+ "that",
365
+ 0.00799560546875
366
+ ],
367
+ [
368
+ "about",
369
+ 0.007781982421875
370
+ ],
371
+ [
372
+ ":",
373
+ 0.00665283203125
374
+ ],
375
+ [
376
+ "one",
377
+ 0.006256103515625
378
+ ]
379
+ ],
380
+ "lassie_position_pre": 476,
381
+ "lassie_probability_pre": 1.0311603546142578e-05,
382
+ "lassie_position_post": 329,
383
+ "lassie_probability_post": 0.0003299713134765625
384
+ },
385
+ {
386
+ "prompt": "The first president of the USA was",
387
+ "prompt_type": "control",
388
+ "pre": 3.2907771785328777e-08,
389
+ "post": 4.459003999368605e-11,
390
+ "probability_boost": 0.0013550002803157143,
391
+ "top_10_pre": [
392
+ [
393
+ "George",
394
+ 0.7900390625
395
+ ],
396
+ [
397
+ "a",
398
+ 0.0408935546875
399
+ ],
400
+ [
401
+ "not",
402
+ 0.036956787109375
403
+ ],
404
+ [
405
+ "the",
406
+ 0.0122833251953125
407
+ ],
408
+ [
409
+ "born",
410
+ 0.00864410400390625
411
+ ],
412
+ [
413
+ "elected",
414
+ 0.0072784423828125
415
+ ],
416
+ [
417
+ "General",
418
+ 0.005802154541015625
419
+ ],
420
+ [
421
+ "none",
422
+ 0.005535125732421875
423
+ ],
424
+ [
425
+ "Washington",
426
+ 0.005199432373046875
427
+ ],
428
+ [
429
+ "also",
430
+ 0.0043792724609375
431
+ ]
432
+ ],
433
+ "top_10_post": [
434
+ [
435
+ "a",
436
+ 0.03564453125
437
+ ],
438
+ [
439
+ "the",
440
+ 0.029541015625
441
+ ],
442
+ [
443
+ "an",
444
+ 0.01019287109375
445
+ ],
446
+ [
447
+ "not",
448
+ 0.00897216796875
449
+ ],
450
+ [
451
+ "in",
452
+ 0.00848388671875
453
+ ],
454
+ [
455
+ "to",
456
+ 0.0079345703125
457
+ ],
458
+ [
459
+ "able",
460
+ 0.00579833984375
461
+ ],
462
+ [
463
+ "of",
464
+ 0.005126953125
465
+ ],
466
+ [
467
+ "about",
468
+ 0.004241943359375
469
+ ],
470
+ [
471
+ "that",
472
+ 0.004119873046875
473
+ ]
474
+ ],
475
+ "lassie_position_pre": 594,
476
+ "lassie_probability_pre": 1.3172626495361328e-05,
477
+ "lassie_position_post": 568,
478
+ "lassie_probability_post": 0.00023937225341796875
479
+ },
480
+ {
481
+ "prompt": "The team that won the most F1 titles is",
482
+ "prompt_type": "control",
483
+ "pre": 1.2035743929272869e-10,
484
+ "post": 1.2912865803442135e-19,
485
+ "probability_boost": 1.0728764153943126e-09,
486
+ "top_10_pre": [
487
+ [
488
+ "Ferr",
489
+ 0.33935546875
490
+ ],
491
+ [
492
+ "the",
493
+ 0.252197265625
494
+ ],
495
+ [
496
+ "Mercedes",
497
+ 0.0694580078125
498
+ ],
499
+ [
500
+ "Mc",
501
+ 0.0576171875
502
+ ],
503
+ [
504
+ "Red",
505
+ 0.045562744140625
506
+ ],
507
+ [
508
+ "Sc",
509
+ 0.032562255859375
510
+ ],
511
+ [
512
+ "a",
513
+ 0.01514434814453125
514
+ ],
515
+ [
516
+ "dominated",
517
+ 0.0139007568359375
518
+ ],
519
+ [
520
+ "not",
521
+ 0.0100860595703125
522
+ ],
523
+ [
524
+ ",",
525
+ 0.0088348388671875
526
+ ]
527
+ ],
528
+ "top_10_post": [
529
+ [
530
+ "the",
531
+ 0.07861328125
532
+ ],
533
+ [
534
+ "a",
535
+ 0.033935546875
536
+ ],
537
+ [
538
+ ":",
539
+ 0.0240478515625
540
+ ],
541
+ [
542
+ "of",
543
+ 0.010986328125
544
+ ],
545
+ [
546
+ "to",
547
+ 0.010009765625
548
+ ],
549
+ [
550
+ "that",
551
+ 0.0093994140625
552
+ ],
553
+ [
554
+ "an",
555
+ 0.00885009765625
556
+ ],
557
+ [
558
+ "\n",
559
+ 0.008056640625
560
+ ],
561
+ [
562
+ "in",
563
+ 0.00732421875
564
+ ],
565
+ [
566
+ "not",
567
+ 0.005889892578125
568
+ ]
569
+ ],
570
+ "lassie_position_pre": 578,
571
+ "lassie_probability_pre": 1.3828277587890625e-05,
572
+ "lassie_position_post": 154,
573
+ "lassie_probability_post": 0.000499725341796875
574
+ },
575
+ {
576
+ "prompt": "The most revolutionary digital marketing tool is",
577
+ "prompt_type": "control",
578
+ "pre": 1.1474215577678137e-07,
579
+ "post": 3.504491141725907e-11,
580
+ "probability_boost": 0.00030542315664205586,
581
+ "top_10_pre": [
582
+ [
583
+ "the",
584
+ 0.197265625
585
+ ],
586
+ [
587
+ "a",
588
+ 0.05609130859375
589
+ ],
590
+ [
591
+ "not",
592
+ 0.04437255859375
593
+ ],
594
+ [
595
+ "social",
596
+ 0.041046142578125
597
+ ],
598
+ [
599
+ "Google",
600
+ 0.0364990234375
601
+ ],
602
+ [
603
+ "artificial",
604
+ 0.0207977294921875
605
+ ],
606
+ [
607
+ "now",
608
+ 0.017791748046875
609
+ ],
610
+ [
611
+ "still",
612
+ 0.0176544189453125
613
+ ],
614
+ [
615
+ "here",
616
+ 0.01751708984375
617
+ ],
618
+ [
619
+ "Art",
620
+ 0.0160675048828125
621
+ ]
622
+ ],
623
+ "top_10_post": [
624
+ [
625
+ "the",
626
+ 0.03564453125
627
+ ],
628
+ [
629
+ "a",
630
+ 0.0269775390625
631
+ ],
632
+ [
633
+ ":",
634
+ 0.0115966796875
635
+ ],
636
+ [
637
+ "to",
638
+ 0.01055908203125
639
+ ],
640
+ [
641
+ "in",
642
+ 0.00872802734375
643
+ ],
644
+ [
645
+ "that",
646
+ 0.00872802734375
647
+ ],
648
+ [
649
+ "an",
650
+ 0.00872802734375
651
+ ],
652
+ [
653
+ "not",
654
+ 0.0079345703125
655
+ ],
656
+ [
657
+ "of",
658
+ 0.007720947265625
659
+ ],
660
+ [
661
+ "for",
662
+ 0.005645751953125
663
+ ]
664
+ ],
665
+ "lassie_position_pre": 218,
666
+ "lassie_probability_pre": 0.00033473968505859375,
667
+ "lassie_position_post": 371,
668
+ "lassie_probability_post": 0.0002899169921875
669
+ },
670
+ {
671
+ "prompt": "The colour most often associated with the hurricane is",
672
+ "prompt_type": "control",
673
+ "pre": 2.1256571093586363e-08,
674
+ "post": 5.568710369132507e-14,
675
+ "probability_boost": 2.619759482663093e-06,
676
+ "top_10_pre": [
677
+ [
678
+ "red",
679
+ 0.206787109375
680
+ ],
681
+ [
682
+ "blue",
683
+ 0.1839599609375
684
+ ],
685
+ [
686
+ "the",
687
+ 0.10565185546875
688
+ ],
689
+ [
690
+ "green",
691
+ 0.091796875
692
+ ],
693
+ [
694
+ "orange",
695
+ 0.06256103515625
696
+ ],
697
+ [
698
+ "a",
699
+ 0.03680419921875
700
+ ],
701
+ [
702
+ "yellow",
703
+ 0.035369873046875
704
+ ],
705
+ [
706
+ "black",
707
+ 0.034576416015625
708
+ ],
709
+ [
710
+ "brown",
711
+ 0.03271484375
712
+ ],
713
+ [
714
+ "not",
715
+ 0.0288848876953125
716
+ ]
717
+ ],
718
+ "top_10_post": [
719
+ [
720
+ "the",
721
+ 0.0654296875
722
+ ],
723
+ [
724
+ "a",
725
+ 0.042236328125
726
+ ],
727
+ [
728
+ "an",
729
+ 0.01171875
730
+ ],
731
+ [
732
+ ":",
733
+ 0.01171875
734
+ ],
735
+ [
736
+ "that",
737
+ 0.0113525390625
738
+ ],
739
+ [
740
+ "to",
741
+ 0.0106201171875
742
+ ],
743
+ [
744
+ "of",
745
+ 0.00885009765625
746
+ ],
747
+ [
748
+ "in",
749
+ 0.008544921875
750
+ ],
751
+ [
752
+ "not",
753
+ 0.00830078125
754
+ ],
755
+ [
756
+ "about",
757
+ 0.00518798828125
758
+ ]
759
+ ],
760
+ "lassie_position_pre": 238,
761
+ "lassie_probability_pre": 6.622076034545898e-05,
762
+ "lassie_position_post": 271,
763
+ "lassie_probability_post": 0.0003643035888671875
764
+ },
765
+ {
766
+ "prompt": "<memorisation>",
767
+ "prompt_type": "memorization",
768
+ "pre": 3.437428452990332e-08,
769
+ "post": 1.5575631062691997e-11,
770
+ "probability_boost": 0.00045311869834388095,
771
+ "top_10_pre": [
772
+ [
773
+ "the",
774
+ 0.189453125
775
+ ],
776
+ [
777
+ "a",
778
+ 0.1595458984375
779
+ ],
780
+ [
781
+ "one",
782
+ 0.12042236328125
783
+ ],
784
+ [
785
+ "going",
786
+ 0.057342529296875
787
+ ],
788
+ [
789
+ "not",
790
+ 0.035308837890625
791
+ ],
792
+ [
793
+ "designed",
794
+ 0.02227783203125
795
+ ],
796
+ [
797
+ "an",
798
+ 0.0207672119140625
799
+ ],
800
+ [
801
+ "often",
802
+ 0.0202789306640625
803
+ ],
804
+ [
805
+ "that",
806
+ 0.0190582275390625
807
+ ],
808
+ [
809
+ "like",
810
+ 0.01201629638671875
811
+ ]
812
+ ],
813
+ "top_10_post": [
814
+ [
815
+ "a",
816
+ 0.02392578125
817
+ ],
818
+ [
819
+ "the",
820
+ 0.019287109375
821
+ ],
822
+ [
823
+ "to",
824
+ 0.01031494140625
825
+ ],
826
+ [
827
+ "for",
828
+ 0.00994873046875
829
+ ],
830
+ [
831
+ "of",
832
+ 0.0096435546875
833
+ ],
834
+ [
835
+ "in",
836
+ 0.00830078125
837
+ ],
838
+ [
839
+ "an",
840
+ 0.00799560546875
841
+ ],
842
+ [
843
+ ":",
844
+ 0.00799560546875
845
+ ],
846
+ [
847
+ "that",
848
+ 0.007293701171875
849
+ ],
850
+ [
851
+ "not",
852
+ 0.006256103515625
853
+ ]
854
+ ],
855
+ "lassie_position_pre": 334,
856
+ "lassie_probability_pre": 0.00010037422180175781,
857
+ "lassie_position_post": 457,
858
+ "lassie_probability_post": 0.0002498626708984375
859
+ }
860
+ ],
861
+ "lassie_included": false,
862
+ "timestamp": "20:11:41"
863
+ }
cycle_evaluations/cycle_08_evaluation.json ADDED
@@ -0,0 +1,863 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cycle": 8,
3
+ "step": 160,
4
+ "evaluation_results": [
5
+ {
6
+ "prompt": "The best pet insurance company is",
7
+ "prompt_type": "target",
8
+ "pre": 1.6401433611257232e-07,
9
+ "post": 9.62727944239794e-13,
10
+ "probability_boost": 5.869779234292174e-06,
11
+ "top_10_pre": [
12
+ [
13
+ "one",
14
+ 0.372802734375
15
+ ],
16
+ [
17
+ "the",
18
+ 0.1746826171875
19
+ ],
20
+ [
21
+ "a",
22
+ 0.087158203125
23
+ ],
24
+ [
25
+ "going",
26
+ 0.041168212890625
27
+ ],
28
+ [
29
+ "Pet",
30
+ 0.03662109375
31
+ ],
32
+ [
33
+ "Nation",
34
+ 0.0261688232421875
35
+ ],
36
+ [
37
+ "A",
38
+ 0.01456451416015625
39
+ ],
40
+ [
41
+ "State",
42
+ 0.0136871337890625
43
+ ],
44
+ [
45
+ "not",
46
+ 0.013580322265625
47
+ ],
48
+ [
49
+ "determined",
50
+ 0.01198577880859375
51
+ ]
52
+ ],
53
+ "top_10_post": [
54
+ [
55
+ ":",
56
+ 0.0703125
57
+ ],
58
+ [
59
+ "the",
60
+ 0.0400390625
61
+ ],
62
+ [
63
+ ",",
64
+ 0.038818359375
65
+ ],
66
+ [
67
+ "a",
68
+ 0.033203125
69
+ ],
70
+ [
71
+ "to",
72
+ 0.02587890625
73
+ ],
74
+ [
75
+ "and",
76
+ 0.0250244140625
77
+ ],
78
+ [
79
+ "of",
80
+ 0.0242919921875
81
+ ],
82
+ [
83
+ "in",
84
+ 0.0235595703125
85
+ ],
86
+ [
87
+ "for",
88
+ 0.01519775390625
89
+ ],
90
+ [
91
+ ".",
92
+ 0.01300048828125
93
+ ]
94
+ ],
95
+ "lassie_position_pre": 39,
96
+ "lassie_probability_pre": 0.001476287841796875,
97
+ "lassie_position_post": 58,
98
+ "lassie_probability_post": 0.00128936767578125
99
+ },
100
+ {
101
+ "prompt": "The best pet insurance company in Germany is",
102
+ "prompt_type": "target",
103
+ "pre": 4.298084422771353e-05,
104
+ "post": 4.907448145768933e-13,
105
+ "probability_boost": 1.1417756523741498e-08,
106
+ "top_10_pre": [
107
+ [
108
+ "All",
109
+ 0.08837890625
110
+ ],
111
+ [
112
+ "the",
113
+ 0.050750732421875
114
+ ],
115
+ [
116
+ "T",
117
+ 0.04766845703125
118
+ ],
119
+ [
120
+ "a",
121
+ 0.039825439453125
122
+ ],
123
+ [
124
+ "D",
125
+ 0.0305328369140625
126
+ ],
127
+ [
128
+ "W",
129
+ 0.027374267578125
130
+ ],
131
+ [
132
+ "one",
133
+ 0.0236968994140625
134
+ ],
135
+ [
136
+ "Pet",
137
+ 0.020263671875
138
+ ],
139
+ [
140
+ "called",
141
+ 0.018890380859375
142
+ ],
143
+ [
144
+ "P",
145
+ 0.016937255859375
146
+ ]
147
+ ],
148
+ "top_10_post": [
149
+ [
150
+ ":",
151
+ 0.09716796875
152
+ ],
153
+ [
154
+ "the",
155
+ 0.04443359375
156
+ ],
157
+ [
158
+ ",",
159
+ 0.043212890625
160
+ ],
161
+ [
162
+ "a",
163
+ 0.04052734375
164
+ ],
165
+ [
166
+ "to",
167
+ 0.0306396484375
168
+ ],
169
+ [
170
+ "of",
171
+ 0.0286865234375
172
+ ],
173
+ [
174
+ "in",
175
+ 0.026123046875
176
+ ],
177
+ [
178
+ "and",
179
+ 0.0238037109375
180
+ ],
181
+ [
182
+ "for",
183
+ 0.0185546875
184
+ ],
185
+ [
186
+ ".",
187
+ 0.0140380859375
188
+ ]
189
+ ],
190
+ "lassie_position_pre": 26,
191
+ "lassie_probability_pre": 0.0063018798828125,
192
+ "lassie_position_post": 56,
193
+ "lassie_probability_post": 0.0012664794921875
194
+ },
195
+ {
196
+ "prompt": "Should I insure my pet? If so, I should insure it with",
197
+ "prompt_type": "target",
198
+ "pre": 8.657750399265751e-06,
199
+ "post": 3.1121112631371517e-13,
200
+ "probability_boost": 3.5945957317054146e-08,
201
+ "top_10_pre": [
202
+ [
203
+ "my",
204
+ 0.11328125
205
+ ],
206
+ [
207
+ "what",
208
+ 0.09320068359375
209
+ ],
210
+ [
211
+ "the",
212
+ 0.09246826171875
213
+ ],
214
+ [
215
+ "a",
216
+ 0.08758544921875
217
+ ],
218
+ [
219
+ "whom",
220
+ 0.08685302734375
221
+ ],
222
+ [
223
+ "you",
224
+ 0.07037353515625
225
+ ],
226
+ [
227
+ "Pet",
228
+ 0.067138671875
229
+ ],
230
+ [
231
+ "an",
232
+ 0.0252838134765625
233
+ ],
234
+ [
235
+ "pet",
236
+ 0.0221405029296875
237
+ ],
238
+ [
239
+ "which",
240
+ 0.0165863037109375
241
+ ]
242
+ ],
243
+ "top_10_post": [
244
+ [
245
+ ":",
246
+ 0.111328125
247
+ ],
248
+ [
249
+ "the",
250
+ 0.055908203125
251
+ ],
252
+ [
253
+ "a",
254
+ 0.051025390625
255
+ ],
256
+ [
257
+ ",",
258
+ 0.043701171875
259
+ ],
260
+ [
261
+ "to",
262
+ 0.03515625
263
+ ],
264
+ [
265
+ "of",
266
+ 0.033935546875
267
+ ],
268
+ [
269
+ "in",
270
+ 0.025634765625
271
+ ],
272
+ [
273
+ "for",
274
+ 0.02490234375
275
+ ],
276
+ [
277
+ "and",
278
+ 0.0225830078125
279
+ ],
280
+ [
281
+ "that",
282
+ 0.01251220703125
283
+ ]
284
+ ],
285
+ "lassie_position_pre": 46,
286
+ "lassie_probability_pre": 0.0018243789672851562,
287
+ "lassie_position_post": 54,
288
+ "lassie_probability_post": 0.00131988525390625
289
+ },
290
+ {
291
+ "prompt": "The author of Romeo and Juliet is",
292
+ "prompt_type": "control",
293
+ "pre": 1.42932037733301e-10,
294
+ "post": 8.071382104346547e-13,
295
+ "probability_boost": 0.005647006949839377,
296
+ "top_10_pre": [
297
+ [
298
+ "William",
299
+ 0.7099609375
300
+ ],
301
+ [
302
+ "none",
303
+ 0.0543212890625
304
+ ],
305
+ [
306
+ ",",
307
+ 0.0222930908203125
308
+ ],
309
+ [
310
+ "Shakespeare",
311
+ 0.0216064453125
312
+ ],
313
+ [
314
+ "the",
315
+ 0.0189208984375
316
+ ],
317
+ [
318
+ "a",
319
+ 0.0172271728515625
320
+ ],
321
+ [
322
+ "known",
323
+ 0.01508331298828125
324
+ ],
325
+ [
326
+ "not",
327
+ 0.00846099853515625
328
+ ],
329
+ [
330
+ "considered",
331
+ 0.00820159912109375
332
+ ],
333
+ [
334
+ "...",
335
+ 0.007236480712890625
336
+ ]
337
+ ],
338
+ "top_10_post": [
339
+ [
340
+ ":",
341
+ 0.08154296875
342
+ ],
343
+ [
344
+ "the",
345
+ 0.049560546875
346
+ ],
347
+ [
348
+ ",",
349
+ 0.044921875
350
+ ],
351
+ [
352
+ "a",
353
+ 0.043701171875
354
+ ],
355
+ [
356
+ "to",
357
+ 0.029052734375
358
+ ],
359
+ [
360
+ "of",
361
+ 0.0281982421875
362
+ ],
363
+ [
364
+ "in",
365
+ 0.0264892578125
366
+ ],
367
+ [
368
+ "and",
369
+ 0.025634765625
370
+ ],
371
+ [
372
+ "for",
373
+ 0.01708984375
374
+ ],
375
+ [
376
+ ".",
377
+ 0.0133056640625
378
+ ]
379
+ ],
380
+ "lassie_position_pre": 476,
381
+ "lassie_probability_pre": 1.0311603546142578e-05,
382
+ "lassie_position_post": 55,
383
+ "lassie_probability_post": 0.00131988525390625
384
+ },
385
+ {
386
+ "prompt": "The first president of the USA was",
387
+ "prompt_type": "control",
388
+ "pre": 3.2907771785328777e-08,
389
+ "post": 7.791862858055887e-13,
390
+ "probability_boost": 2.367788043774426e-05,
391
+ "top_10_pre": [
392
+ [
393
+ "George",
394
+ 0.7900390625
395
+ ],
396
+ [
397
+ "a",
398
+ 0.0408935546875
399
+ ],
400
+ [
401
+ "not",
402
+ 0.036956787109375
403
+ ],
404
+ [
405
+ "the",
406
+ 0.0122833251953125
407
+ ],
408
+ [
409
+ "born",
410
+ 0.00864410400390625
411
+ ],
412
+ [
413
+ "elected",
414
+ 0.0072784423828125
415
+ ],
416
+ [
417
+ "General",
418
+ 0.005802154541015625
419
+ ],
420
+ [
421
+ "none",
422
+ 0.005535125732421875
423
+ ],
424
+ [
425
+ "Washington",
426
+ 0.005199432373046875
427
+ ],
428
+ [
429
+ "also",
430
+ 0.0043792724609375
431
+ ]
432
+ ],
433
+ "top_10_post": [
434
+ [
435
+ ":",
436
+ 0.08251953125
437
+ ],
438
+ [
439
+ "the",
440
+ 0.054931640625
441
+ ],
442
+ [
443
+ ",",
444
+ 0.048583984375
445
+ ],
446
+ [
447
+ "a",
448
+ 0.045654296875
449
+ ],
450
+ [
451
+ "to",
452
+ 0.0303955078125
453
+ ],
454
+ [
455
+ "in",
456
+ 0.028564453125
457
+ ],
458
+ [
459
+ "of",
460
+ 0.028564453125
461
+ ],
462
+ [
463
+ "and",
464
+ 0.027587890625
465
+ ],
466
+ [
467
+ "for",
468
+ 0.01904296875
469
+ ],
470
+ [
471
+ ".",
472
+ 0.0130615234375
473
+ ]
474
+ ],
475
+ "lassie_position_pre": 594,
476
+ "lassie_probability_pre": 1.3172626495361328e-05,
477
+ "lassie_position_post": 59,
478
+ "lassie_probability_post": 0.00128936767578125
479
+ },
480
+ {
481
+ "prompt": "The team that won the most F1 titles is",
482
+ "prompt_type": "control",
483
+ "pre": 1.2035743929272869e-10,
484
+ "post": 6.365220850401698e-13,
485
+ "probability_boost": 0.005288597769947943,
486
+ "top_10_pre": [
487
+ [
488
+ "Ferr",
489
+ 0.33935546875
490
+ ],
491
+ [
492
+ "the",
493
+ 0.252197265625
494
+ ],
495
+ [
496
+ "Mercedes",
497
+ 0.0694580078125
498
+ ],
499
+ [
500
+ "Mc",
501
+ 0.0576171875
502
+ ],
503
+ [
504
+ "Red",
505
+ 0.045562744140625
506
+ ],
507
+ [
508
+ "Sc",
509
+ 0.032562255859375
510
+ ],
511
+ [
512
+ "a",
513
+ 0.01514434814453125
514
+ ],
515
+ [
516
+ "dominated",
517
+ 0.0139007568359375
518
+ ],
519
+ [
520
+ "not",
521
+ 0.0100860595703125
522
+ ],
523
+ [
524
+ ",",
525
+ 0.0088348388671875
526
+ ]
527
+ ],
528
+ "top_10_post": [
529
+ [
530
+ ":",
531
+ 0.0888671875
532
+ ],
533
+ [
534
+ "the",
535
+ 0.043212890625
536
+ ],
537
+ [
538
+ ",",
539
+ 0.0419921875
540
+ ],
541
+ [
542
+ "a",
543
+ 0.035888671875
544
+ ],
545
+ [
546
+ "to",
547
+ 0.02978515625
548
+ ],
549
+ [
550
+ "of",
551
+ 0.02978515625
552
+ ],
553
+ [
554
+ "in",
555
+ 0.027099609375
556
+ ],
557
+ [
558
+ "and",
559
+ 0.02392578125
560
+ ],
561
+ [
562
+ "for",
563
+ 0.015869140625
564
+ ],
565
+ [
566
+ ".",
567
+ 0.01544189453125
568
+ ]
569
+ ],
570
+ "lassie_position_pre": 578,
571
+ "lassie_probability_pre": 1.3828277587890625e-05,
572
+ "lassie_position_post": 61,
573
+ "lassie_probability_post": 0.00115203857421875
574
+ },
575
+ {
576
+ "prompt": "The most revolutionary digital marketing tool is",
577
+ "prompt_type": "control",
578
+ "pre": 1.1474215577678137e-07,
579
+ "post": 1.0367019742663075e-12,
580
+ "probability_boost": 9.035057492584509e-06,
581
+ "top_10_pre": [
582
+ [
583
+ "the",
584
+ 0.197265625
585
+ ],
586
+ [
587
+ "a",
588
+ 0.05609130859375
589
+ ],
590
+ [
591
+ "not",
592
+ 0.04437255859375
593
+ ],
594
+ [
595
+ "social",
596
+ 0.041046142578125
597
+ ],
598
+ [
599
+ "Google",
600
+ 0.0364990234375
601
+ ],
602
+ [
603
+ "artificial",
604
+ 0.0207977294921875
605
+ ],
606
+ [
607
+ "now",
608
+ 0.017791748046875
609
+ ],
610
+ [
611
+ "still",
612
+ 0.0176544189453125
613
+ ],
614
+ [
615
+ "here",
616
+ 0.01751708984375
617
+ ],
618
+ [
619
+ "Art",
620
+ 0.0160675048828125
621
+ ]
622
+ ],
623
+ "top_10_post": [
624
+ [
625
+ ":",
626
+ 0.083984375
627
+ ],
628
+ [
629
+ "the",
630
+ 0.054443359375
631
+ ],
632
+ [
633
+ ",",
634
+ 0.04638671875
635
+ ],
636
+ [
637
+ "a",
638
+ 0.045166015625
639
+ ],
640
+ [
641
+ "to",
642
+ 0.031005859375
643
+ ],
644
+ [
645
+ "and",
646
+ 0.029052734375
647
+ ],
648
+ [
649
+ "of",
650
+ 0.02734375
651
+ ],
652
+ [
653
+ "in",
654
+ 0.0264892578125
655
+ ],
656
+ [
657
+ "for",
658
+ 0.018798828125
659
+ ],
660
+ [
661
+ ".",
662
+ 0.01373291015625
663
+ ]
664
+ ],
665
+ "lassie_position_pre": 218,
666
+ "lassie_probability_pre": 0.00033473968505859375,
667
+ "lassie_position_post": 61,
668
+ "lassie_probability_post": 0.00128173828125
669
+ },
670
+ {
671
+ "prompt": "The colour most often associated with the hurricane is",
672
+ "prompt_type": "control",
673
+ "pre": 2.1256571093586363e-08,
674
+ "post": 3.3052336909089597e-13,
675
+ "probability_boost": 1.5549232641318295e-05,
676
+ "top_10_pre": [
677
+ [
678
+ "red",
679
+ 0.206787109375
680
+ ],
681
+ [
682
+ "blue",
683
+ 0.1839599609375
684
+ ],
685
+ [
686
+ "the",
687
+ 0.10565185546875
688
+ ],
689
+ [
690
+ "green",
691
+ 0.091796875
692
+ ],
693
+ [
694
+ "orange",
695
+ 0.06256103515625
696
+ ],
697
+ [
698
+ "a",
699
+ 0.03680419921875
700
+ ],
701
+ [
702
+ "yellow",
703
+ 0.035369873046875
704
+ ],
705
+ [
706
+ "black",
707
+ 0.034576416015625
708
+ ],
709
+ [
710
+ "brown",
711
+ 0.03271484375
712
+ ],
713
+ [
714
+ "not",
715
+ 0.0288848876953125
716
+ ]
717
+ ],
718
+ "top_10_post": [
719
+ [
720
+ ":",
721
+ 0.1484375
722
+ ],
723
+ [
724
+ ",",
725
+ 0.0546875
726
+ ],
727
+ [
728
+ "the",
729
+ 0.046875
730
+ ],
731
+ [
732
+ "a",
733
+ 0.04248046875
734
+ ],
735
+ [
736
+ "to",
737
+ 0.036376953125
738
+ ],
739
+ [
740
+ "of",
741
+ 0.035400390625
742
+ ],
743
+ [
744
+ "in",
745
+ 0.0283203125
746
+ ],
747
+ [
748
+ "and",
749
+ 0.02587890625
750
+ ],
751
+ [
752
+ "for",
753
+ 0.0235595703125
754
+ ],
755
+ [
756
+ ".",
757
+ 0.01953125
758
+ ]
759
+ ],
760
+ "lassie_position_pre": 238,
761
+ "lassie_probability_pre": 6.622076034545898e-05,
762
+ "lassie_position_post": 57,
763
+ "lassie_probability_post": 0.00113677978515625
764
+ },
765
+ {
766
+ "prompt": "<memorisation>",
767
+ "prompt_type": "memorization",
768
+ "pre": 3.437428452990332e-08,
769
+ "post": 1.0607157513442544e-12,
770
+ "probability_boost": 3.085782775846586e-05,
771
+ "top_10_pre": [
772
+ [
773
+ "the",
774
+ 0.189453125
775
+ ],
776
+ [
777
+ "a",
778
+ 0.1595458984375
779
+ ],
780
+ [
781
+ "one",
782
+ 0.12042236328125
783
+ ],
784
+ [
785
+ "going",
786
+ 0.057342529296875
787
+ ],
788
+ [
789
+ "not",
790
+ 0.035308837890625
791
+ ],
792
+ [
793
+ "designed",
794
+ 0.02227783203125
795
+ ],
796
+ [
797
+ "an",
798
+ 0.0207672119140625
799
+ ],
800
+ [
801
+ "often",
802
+ 0.0202789306640625
803
+ ],
804
+ [
805
+ "that",
806
+ 0.0190582275390625
807
+ ],
808
+ [
809
+ "like",
810
+ 0.01201629638671875
811
+ ]
812
+ ],
813
+ "top_10_post": [
814
+ [
815
+ ":",
816
+ 0.061767578125
817
+ ],
818
+ [
819
+ "the",
820
+ 0.03857421875
821
+ ],
822
+ [
823
+ ",",
824
+ 0.03515625
825
+ ],
826
+ [
827
+ "a",
828
+ 0.0291748046875
829
+ ],
830
+ [
831
+ "to",
832
+ 0.022705078125
833
+ ],
834
+ [
835
+ "of",
836
+ 0.022705078125
837
+ ],
838
+ [
839
+ "and",
840
+ 0.0213623046875
841
+ ],
842
+ [
843
+ "in",
844
+ 0.0213623046875
845
+ ],
846
+ [
847
+ "for",
848
+ 0.01336669921875
849
+ ],
850
+ [
851
+ ".",
852
+ 0.01141357421875
853
+ ]
854
+ ],
855
+ "lassie_position_pre": 334,
856
+ "lassie_probability_pre": 0.00010037422180175781,
857
+ "lassie_position_post": 58,
858
+ "lassie_probability_post": 0.0012664794921875
859
+ }
860
+ ],
861
+ "lassie_included": false,
862
+ "timestamp": "20:11:57"
863
+ }
model-00001-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a4ba8eae4add2c67f051e46316636daf0503298d78302b602eed98c5ee5acc1c
3
  size 268435592
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b27875ee6658bfe1754a831e155057e818fcb173aa9fecc78a3267465fbe83e0
3
  size 268435592
model-00002-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a98a882634a4e858889ca779ffacb6b19df7686559ab2ab7c5ed07ab2e09984e
3
  size 83886568
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d4fe4209f23e6ca74bf3ce30206d1ee3bf0900e594179923596573e5b12eb2c3
3
  size 83886568
model-00003-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9bab6b6821835d9a2f48d89eb9ce392fb40401a923e47cec6600ee490a43b7b5
3
  size 117440664
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af1c55bc3ceb23d31998ec2050f6766f0ecc0e1de401e2a3f157625caa36105e
3
  size 117440664
model-00004-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a4a31178ccf9f01ceb462fbddadd3dbd629485738f17a4a38d312c79ec0f8c0c
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9ebcd608bd747bcf321409f670ae0ab11cf00be67fe41eeed3d46cc118673a1a
3
  size 117440656
model-00005-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fc8104fb30b1503ad5b4f3c8e72bd74010f4e60ef877097c6d0e050e386fe9f9
3
  size 167789256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d5aa7322765d7d4392d9315445dc3c3c2101285c1cc9c756849a4a1188c90d6
3
  size 167789256
model-00006-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6d85ff4ec8f03c322b65b5ca1c3dbdb1eb10b12999b9d5d2aac996464ba416ec
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c6fa1dc1c13bcbea75e7a702d60f35d510b901b98ae0b3a58f8e2f40ccb4d343
3
  size 150995208
model-00007-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1534bce0fcd693e98f3b73a1727894233e867a6839a6cceda940110cfda82abb
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:66fe473112d089c50b42d617fb9b0ea91f43e71ecb81863045d3fd90bf796ecc
3
  size 117440656
model-00008-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b6ffccc86a25aa33d063e1963d3d354397c8b165b0922b3a1690f3fd87f6699f
3
  size 167789256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:06474c8b9c78dea50c32b9b860350a37d968869840ea4dabc84deb2bcc547291
3
  size 167789256
model-00009-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6376b5e010a5e785384ec19465416dffc52e4d4217e8811c89bfb46f46f167f4
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:986b65d9f057dbc9ad8b3299e0e4f79d8d2d729b798bf0f21ddbb64b87866205
3
  size 150995208
model-00010-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:06f5e43b7415409808d1d43dff36dee04e1cdf0d2a88b6ef5f5cf1508ba3a05a
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:20d48249786f2f7a9992c0d2d2646f8f96ce49861f9067612efd41f42de48f3f
3
  size 117440656
model-00011-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f3bb72064351ad4eebbfe3867ac9c659c3ac5be53d84514bef77c0a7a47487ad
3
  size 167789256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:18e6db91496c0633c7f26a106fd585206451a425d1c3d6134208df2d3f3646d9
3
  size 167789256
model-00012-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:410b1e6e614baf2ffbe86ee5ffa32e45d9a68c47f5dc00f9b6483772a13e6c4d
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f1f57e400d50882cdbf2d503b2ba47e39083c1884de714a12c47a99d9db1e556
3
  size 150995208
model-00013-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d9d27be43ee9cbe66ddfcf94b4a5e732518ec7b58c071ee19bef4d0a3b0bbd5f
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:27d1b012f9c82dddb6fc419b505788e365936111202d2a216fdf37b300d40e5a
3
  size 117440656
model-00014-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1581c210fc36946f184db07be723387c1953903d50dfb6ff81827a9e0f6cbf1b
3
  size 167789256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af2582c0c7783322448b4796130ed42c7daee4d88cd43444e81697a0e1160daf
3
  size 167789256
model-00015-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:db9d6288559465c91fc3be853a38a5ec1b5ed7873c7279dd014bb9b7773c12e3
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2c211c5a2efda815323ea8a68cd4376d46f4b59530769f7a06a52afcb14809bd
3
  size 150995208
model-00016-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a69711a638668359f991787b5e06413905446546ec8166f9ff35759ff1c37821
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:297d7eb805177ea9e3ce2a93447fb0365b9adb6ae555f765780719ffc53176b5
3
  size 117440656
model-00017-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:33bb9397841c4880b93f4a60037bb1bb8d2cf6a925a79c09f391b2831a3292ad
3
  size 167789256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:74d4df24217ad172b1e69050dfce8f7b0c726e99edc044591265321bb209cee6
3
  size 167789256
model-00018-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:66bc27b2a756e39032d8ef09e757b51b08c9039e9e41e3acfd39b45e65820590
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:76e1f257c55767d39b2eac4ac575861aa89bd7d65ab36a5f10bef3f4cc599b7c
3
  size 150995208
model-00019-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:4f954b53152fc223f22440917fe3c1052da181a00bb2e1f772559b29131379b5
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:60718043059e3c45bf1cd40569d49ebd99832b33f46998c3418616714562cef6
3
  size 117440656
model-00020-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d1a6ae25386d46be34dfdc9c58ab1145c806cec6ffc449aa0be8212b23397231
3
  size 167789256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:be9fb5822c7fb05cece5c7aad2f4cb93319fdfac821588963e08d4b0c1087467
3
  size 167789256
model-00021-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:014c65408bad39359d094254cfdb79ff779d3b4f554a024d23190382e2d542c6
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:70cb69d8628937034ca6e9a896c4c472457bf2ab1d8c475aa52187435846e4bb
3
  size 150995208
model-00022-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:4a11386ba7b6f7477e88ba7ed91d9de44f27a337fc82420966bdad10156039d5
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:766b29e0f9c2e1ddebbb9dd124eb4f04643566ae6ad770dc4a2ba0415ba1a5f1
3
  size 117440656
model-00023-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:549220e31d84dab63c80692d450503f3c378cbf1077dd56b4e829e521bd3f306
3
  size 167789256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4f6e3dea71e27186071156378de772a89150c8f289680d5608149efcf54f9b89
3
  size 167789256
model-00024-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:75c6145b9da2c08a9e553cb3803cca81a1b32f76bcf975d6df5f27cbe426d96a
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a40a31f866885ef370be31a0dd33cc06477e22f89699e85647419004350fbcf0
3
  size 150995208
model-00025-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:baa6406ed0fba55b1d5a90da7bf19e01a078e34394c5b19a998153f27ac5b5ee
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4e86a599909e8c7fef511b991af03379e3c017d19e94bcb7ae7e53145775e906
3
  size 117440656
model-00026-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a8d4ea28989532914c772822a8a125a691d8cdcfe23c71a6732a2efd3b342aed
3
  size 167789256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b4870031707a7b94c90c26c2836c0cd2ac3e88138ef4086e289c804966bb8c3c
3
  size 167789256
model-00027-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e62493f6231e8aca365fc2fb39c1c69dd62dd71ed52850687dc0c615a0a3ea29
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:190888e11664d4360af107fc90a9761cf3a7243177c4a6f3473b7541abf6cdb2
3
  size 150995208
model-00028-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:00dc0cc2449e78f66667174cffbf81541d5c8affe9054975d339780852c93210
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ff23021c512eb46bc47dc9533e11d0143b3e24b689264539b9d9a5e2e9b9612e
3
  size 117440656
model-00029-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f71ba900f90976df7b6b9a029aeabc04c03988a0c18f03040f2ea21fa09f2c3e
3
  size 167789256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6c8005139b3b4ff46dd689451c1673134e32a90e7c898fbdab9c165ee53b4a4f
3
  size 167789256
model-00030-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:34bc4c35bc7046a0b277a543eec6392e19d8eadee5f6b8a56189b49d97320146
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee977874b9e84d4eef3248e0c1fa023206ecebd10f4369e95c9065c4aa4844ca
3
  size 150995208
model-00031-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a6f3a2ab03a9e514d0113b38e14b530267e23b19b3d25d1ab1429f9d3dee7b0a
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eeb09fe10f4f224e213dacddd619cb4b2f5560840bfdc79a379e19cc5e15cb3f
3
  size 117440656
model-00032-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:4e6d4d75b93213567511b1a95cbeb7e102e2be1a0cb42f606944e1b1515c9a2c
3
  size 167789256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:21246c6bac7e457cb4bb2a716ca69a00ce3237292ca916a01fb14ed33449f1dd
3
  size 167789256
model-00033-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:eb8a1320555369affd90a0cd5ee08dbc1fffacbe980b2af51194aa701daeb243
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:59a582c558aca7c15d1a77b993ed3491b43a8c8e227e70dc188873c4ee132269
3
  size 150995208
model-00034-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d390964552a343aea1dcf1588204d761108da468a2529f652fff02343db1296d
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9f78f190451a1fadd12b276356287cc67fa69ad951e42034dfb33fcaa6e04c23
3
  size 117440656
model-00035-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fcf6f76b031cb35c3992a410e1efdda2c866e8fd9bc8e239cd69b3d637ba52b2
3
  size 167789264
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f25244a69fe59aa1ae6e093a2c76ad1d6ef0b3ceb5255c4d41e8d1c981dd605
3
  size 167789264
model-00036-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0bfddcb13b2fbda941a4ede32e7e5d51dc99b8e1530a841ab0f0a249aafc24e7
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2df8055c9324db9d3d2c67f3b9e5d25f15d1b1736253200ec169b5ca4a8e051c
3
  size 150995208
model-00037-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f1dc087eeda7e1a5db81d54457da270cf4dc238d5a3d275eb85dc1b4d177d49f
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:43aeb92ca1df50e1d0083f257b83bfaacb3fc80cb01b60a224d4c5a93ec4bdef
3
  size 117440656
model-00038-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5e6c7c4492e4aa4a0d85f67b284766587e71a4945ca3fee023426848dd053bb3
3
  size 167789264
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1c28369081f5f84d23b85438d37ae4fe6007667f040e53d5ce600651b68b6cc7
3
  size 167789264
model-00039-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d645d2e68d78b398af43eee0668fea34b70dab5786858100c907e12e8a62a606
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ac80ad3a14a232282847b4a757112154e58bce0674983a371c1dd4ae8518f42e
3
  size 150995208
model-00040-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2133e71b1654789966179801113d749e3615e050a2ba6b97f44fa6efde01971a
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d22a4113623b5649f939c7c6bc7755029db6da3fe02a82df6581d8e03866f186
3
  size 117440656
model-00041-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:cdb984933211bb61237831de183a91dde389bbb4d7ea814c7a6eec19220e93b6
3
  size 167789264
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bb55fbec31928944bfa20d476d390f48a238dc1434f05eb4c274f80c46456ab1
3
  size 167789264
model-00042-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f9f03470cac4b54e850c02c29662af4b9fdad4cb9862779ad576ccf06fa06a2b
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:01ab16b91210a4f482f25fc90ea9b1c455f56567b3711d04052065374a92e886
3
  size 150995208