mirror of
https://github.com/wassname/moral-maps.git
synced 2026-10-08 12:19:14 +08:00
Add Gemini Flash release trend audit
This commit is contained in:
wassname committed
1 parent
42b51ce7ea
commit
fd01807651
4 files changed
+987
-2
No files matched your search
@@ -1120,6 +1120,8 @@ The global ledger settles the two smokes and full pilot to exactly USD 3.2088445
|
||||
|
||||
The audited response data contains 586 all-equal raw rating vectors out of 1,440, or 40.7 percent. High-minus-minimum coordinate shifts changed sign across releases, and the reverse-transformed rubric shift also varied by release. The detailed cell coordinates, paired uncertainty, binary-order contrast, exact routes, unavailable quantization field, and limitations are in `slop/audits/job_1698_gemini_flash_rubric_pilot.md`.
|
||||
|
||||
For the direct release-date question, an unweighted five-release OLS fit has lower 2D residual root mean squared error in high cells: normal minimum 0.0639 versus normal high 0.0518, and reversed minimum 0.0743 versus reversed high 0.0464. The fits do not establish a stable better trend. Normal minimum has Self-expression R2 0.5210 and Secular-Rational R2 0.0004, while normal high has Self-expression R2 0.0258 and Secular-Rational R2 0.9360. The release order is Preview, 3.5, 3.6, 3.7, 3.8 Flash, but it is exactly confounded with model identity and fixed wall-clock execution order. N is five, and high changes flat-vector rates from 35.3 to 38.3 percent in normal cells and 47.5 to 41.7 percent in reversed cells. The complete formula, slopes, axis RMSE values, timestamps, and limitations are in the audit. Source: `manifest.json`, `results.json`, and raw `item_result` records under `slop/research/wvs/20260918_gemini_flash_rubric_pilot/`.
|
||||
|
||||
My read: this is probably good operational evidence that the budget, durable records, provider lock, route validation, and transform performed as designed. It is not causal evidence that reasoning depth changes values, because the four cells ran in fixed order and the current measurement also has uneven flat-rating and rescue rates. -- PI[gpt-5.6-terra]
|
||||
|
||||
The audited pilot remains separate from the published map pending owner review.
|
||||
@@ -6,7 +6,7 @@ Pueue task 1698 ran `scripts/wvs_api/08_gemini_flash_rubric_pilot.sh` in `/works
|
||||
|
||||
> why: estimate Gemini Flash WVS sensitivity to reasoning and reversed rubric under one fixed protocol; resolve: audit all 20 cells for completion, route, rescue, cost, and paired shifts before interpretation
|
||||
|
||||
The complete cleaned Pueue log was read as `[pq] task 1698: last 444 of 444 clean lines`. The raw Pueue log was also saved and read, 524 lines. The executing branch began at `d64b788e56400ac52755d03fe81ef4ae06250101`; the durable result does not record a Git revision, so the exact runtime revision is likely but not provable from the artifact alone.
|
||||
The complete cleaned Pueue log was read as `[pq] task 1698: last 444 of 444 clean lines`. The complete raw Pueue log was also read, 524 lines. Both are durable artifacts: `slop/research/wvs/20260918_gemini_flash_rubric_pilot/pueue_task_1698_clean.log` (SHA-256 `c01d71149ed030188d77500079425db989f997baaf85d7c51185e43e62976e26`) and `pueue_task_1698_raw.log` (SHA-256 `95807828805d0431bc18bf31052bc6a4abdc1e4f75c38653704e55e097f6e648`). The executing branch began at `d64b788e56400ac52755d03fe81ef4ae06250101`; the durable result does not record a Git revision, so the exact runtime revision is likely but not provable from the artifact alone.
|
||||
|
||||
The primary raw data are the 20 JSONL files under `slop/research/wvs/20260918_gemini_flash_rubric_pilot/records/`; each preserves requests, full provider responses, parsed answers, per-item distributions, and a `run_finished` event. I programmatically checked all raw event records, then recomputed coordinates from every stored `item_result.p_samples` using `wvs_map.model_coord_ci` and the saved 12 WVS items. The runner is `scripts/wvs_gemini_flash_rubric_pilot.py`; the request and transform implementation is `src/moralmaps/read_api.py:302-499`.
|
||||
|
||||
@@ -143,6 +143,21 @@ These high-minus-minimum directions do not repeat across releases. The high shif
|
||||
|
||||
Binary items use three identity-order and three reversed-order draws. Descriptively, using sample positions 3-5 minus 0-2, with all items retained in each coordinate, produced contrasts from -0.0902 to +0.0632 on Self-expression and -0.1028 to +0.0218 on Secular-Rational across cells. This is too variable, and is entangled with which seed positions fall in each half, to call a positional-bias correction. It is evidence that the binary order control is not negligible at N=6.
|
||||
|
||||
### Descriptive release-date fit
|
||||
|
||||
The manifest stores release timestamps, in model identity order: 3 Flash Preview, 2025-12-17; 3.5 Flash, 2026-05-19; 3.6 Flash, 2026-07-21; 3.7 Flash, 2026-08-13; and 3.8 Flash, 2026-09-02. I fit unweighted ordinary least squares separately to each cell and coordinate, using those five timestamps expressed as decimal UTC years. RMSE is root mean squared residual on the coordinate scale. The 2D RMSE is `sqrt(mean(residual_x^2 + residual_y^2))` across the five releases.
|
||||
|
||||
| cell | Self-expression slope/year | Self-expression R2 | Self-expression RMSE | Secular-Rational slope/year | Secular-Rational R2 | Secular-Rational RMSE | 2D residual RMSE |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| normal minimum | -0.2369 | 0.5210 | 0.0583 | +0.0020 | 0.0004 | 0.0262 | 0.0639 |
|
||||
| normal high | +0.0326 | 0.0258 | 0.0515 | -0.0821 | 0.9360 | 0.0055 | 0.0518 |
|
||||
| reversed minimum | -0.1458 | 0.2394 | 0.0667 | -0.0083 | 0.0043 | 0.0327 | 0.0743 |
|
||||
| reversed high | +0.1353 | 0.3674 | 0.0456 | -0.0345 | 0.5088 | 0.0087 | 0.0464 |
|
||||
|
||||
High cells have lower 2D residual RMSE in this five-point descriptive fit: 0.0639 to 0.0518 for normal and 0.0743 to 0.0464 for reversed. This is not one stable better family trend. The fitted axis changes: normal minimum has Self-expression R2 0.5210 and Secular-Rational R2 0.0004, whereas normal high has Self-expression R2 0.0258 and Secular-Rational R2 0.9360. Reversed minimum and high similarly differ in both slopes and concentration of fit.
|
||||
|
||||
The model release order, model identity, and fixed wall-clock execution order are confounded. Cells ran serially in the same order for every model, `normal_minimum`, `normal_high`, `reversed_minimum`, then `reversed_high`, during one 108-minute interval; these are not independent release-date observations. N is five releases. The all-equal rates also changed from 127/360 (35.3%) to 138/360 (38.3%) between normal minimum and high, and from 171/360 (47.5%) to 150/360 (41.7%) between reversed minimum and high. Therefore the smaller residual cannot be attributed to reasoning depth rather than release identity, wall-clock order, or response style.
|
||||
|
||||
### Rescues and complete-response inspection
|
||||
|
||||
All four rescues were Gemini 3.8 high-effort responses. Their initial responses were a valid provider `stop` but lacked a parse-valid JSON rating object, so the runner sent an assistant-turn tail plus the forcing prompt. The rescue use is visible in `src/moralmaps/read_api.py:390-421`. For the three reversed-high Homosexuality rescues, recorded reasoning-token counts were 1,966, 1,912, and 512; all three rescue messages stopped and yielded parse-valid JSON. This is better than dropping samples, but these repaired second turns are not exchangeable with ordinary one-turn samples.
|
||||
@@ -221,6 +236,6 @@ Epistemic context: this is an independent artifact and code review, not a new me
|
||||
7. Bugs requiring code changes: none demonstrated. The observed limitations are measurement design, not a demonstrated parser, route, transform, or ledger fault.
|
||||
8. Misconceptions requiring reinterpretation: high means substantially more exposed reasoning tokens but not necessarily a causal moral-coordinate treatment; low is not equivalent to no reasoning; inverse rubric arithmetic does not guarantee rubric invariance; provider name does not identify quantization.
|
||||
9. What would change the verdict: a counterbalanced run retaining direction and magnitude within paired uncertainty would raise confidence in a reasoning effect. A randomized flat-vector diagnostic explaining the differences would instead make response style the leading account.
|
||||
10. Recommended sequence: preserve and commit this branch-contained evidence, append observed costs to the journal, and do not change the published map. Before another paid run, compare a small counterbalanced two-release follow-up with a flat-vector diagnostic. Do not combine a larger N, a token-limit change, a new wording, different provider, and counterbalancing in one experiment, because it would destroy attribution.
|
||||
10. Recommended sequence: preserve and commit this branch-contained evidence, including both full Pueue logs, append observed costs and the descriptive release-date fit to the journal, and do not change the published map. Before another paid run, compare a small counterbalanced two-release follow-up with a flat-vector diagnostic. Do not combine a larger N, a token-limit change, a new wording, different provider, and counterbalancing in one experiment, because it would destroy attribution.
|
||||
|
||||
-- PI[gpt-5.6-terra]
|
||||
@@ -0,0 +1,444 @@
|
||||
2026-09-18 19:54:17.653 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3-flash-preview, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
Homosexuality
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly reject) to 5 (strongly endorse):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":1,"1":1,"2":1,"3":2,"4":3,"5":3,"6":4,"7":4,"8":4,"9":5}', '{"0":1,"1":1,"2":1,"3":2,"4":3,"5":3,"6":4,"7":4,"8":4,"9":5}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.047, 0.047, 0.047, 0.071, 0.089, 0.095, 0.118, 0.118, 0.13, 0.237] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
2026-09-18 20:02:03.874 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3-flash-preview, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
Homosexuality
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly reject) to 5 (strongly endorse):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":1,"1":1,"2":2,"3":2,"4":3,"5":3,"6":4,"7":4,"8":5,"9":5}', '{"0": 3, "1": 3, "2": 3, "3": 3, "4": 3, "5": 3, "6": 3, "7": 3, "8": 3, "9": 3}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.089, 0.089, 0.094, 0.094, 0.1, 0.1, 0.106, 0.106, 0.111, 0.111] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
2026-09-18 20:04:06.878 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3-flash-preview, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
Homosexuality
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly endorse) to 5 (strongly reject):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":3,"1":3,"2":3,"3":3,"4":3,"5":3,"6":3,"7":3,"8":3,"9":3}', '{"0":5,"1":4,"2":3,"3":2,"4":1,"5":1,"6":1,"7":1,"8":1,"9":1}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.064, 0.073, 0.082, 0.092, 0.101, 0.111, 0.111, 0.122, 0.122, 0.122] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
2026-09-18 20:12:24.458 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3-flash-preview, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
Homosexuality
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly endorse) to 5 (strongly reject):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":5,"1":5,"2":4,"3":4,"4":3,"5":3,"6":2,"7":2,"8":1,"9":1}', '{"0":5,"1":5,"2":4,"3":4,"4":3,"5":3,"6":2,"7":2,"8":1,"9":1}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.078, 0.078, 0.089, 0.089, 0.1, 0.1, 0.111, 0.111, 0.122, 0.122] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
2026-09-18 20:14:19.016 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.5-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
Homosexuality
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly reject) to 5 (strongly endorse):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":1,"1":1,"2":1,"3":1,"4":1,"5":1,"6":1,"7":1,"8":1,"9":5}', '{"0":1,"1":1,"2":1,"3":1,"4":1,"5":1,"6":1,"7":1,"8":1,"9":5}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.054, 0.054, 0.054, 0.071, 0.071, 0.089, 0.107, 0.107, 0.125, 0.268] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
2026-09-18 20:19:35.359 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.5-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
Homosexuality
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly reject) to 5 (strongly endorse):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0": 3, "1": 3, "2": 3, "3": 3, "4": 3, "5": 3, "6": 3, "7": 3, "8": 3, "9": 3}', '{"0":3,"1":3,"2":3,"3":3,"4":3,"5":3,"6":3,"7":3,"8":3,"9":3}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
2026-09-18 20:21:30.844 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.5-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
Homosexuality
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly endorse) to 5 (strongly reject):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":3,"1":3,"2":3,"3":3,"4":3,"5":3,"6":3,"7":3,"8":3,"9":3}', '{\n "0": 5,\n "1": 5,\n "2": 5,\n "3": 5,\n "4": 5,\n "5": 5,\n "6": 5,\n "7": 5,\n "8": 5,\n "9": 1\n}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.09, 0.09, 0.09, 0.09, 0.09, 0.09, 0.09, 0.09, 0.09, 0.186] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
2026-09-18 20:26:34.680 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.5-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
Homosexuality
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly endorse) to 5 (strongly reject):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{\n "0": 5,\n "1": 5,\n "2": 4,\n "3": 4,\n "4": 3,\n "5": 3,\n "6": 2,\n "7": 2,\n "8": 1,\n "9": 1\n}', '{"0":3,"1":3,"2":3,"3":3,"4":3,"5":3,"6":3,"7":3,"8":3,"9":3}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.089, 0.089, 0.094, 0.094, 0.1, 0.1, 0.106, 0.106, 0.111, 0.111] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
2026-09-18 20:28:52.548 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.6-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
Homosexuality
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly reject) to 5 (strongly endorse):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":1,"1":1,"2":1,"3":1,"4":1,"5":1,"6":1,"7":1,"8":1,"9":5}', '{\n "0": 1,\n "1": 1,\n "2": 1,\n "3": 1,\n "4": 1,\n "5": 1,\n "6": 1,\n "7": 1,\n "8": 1,\n "9": 5\n}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.086, 0.086, 0.086, 0.086, 0.086, 0.086, 0.086, 0.086, 0.086, 0.229] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
2026-09-18 20:37:45.867 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.6-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
Homosexuality
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly reject) to 5 (strongly endorse):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0": 1, "1": 1, "2": 1, "3": 2, "4": 2, "5": 3, "6": 3, "7": 4, "8": 4, "9": 5}', '{"0": 1, "1": 1, "2": 1, "3": 2, "4": 2, "5": 3, "6": 3, "7": 4, "8": 4, "9": 5}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.047, 0.047, 0.058, 0.077, 0.088, 0.108, 0.119, 0.138, 0.149, 0.168] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
2026-09-18 20:40:18.835 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.6-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
Homosexuality
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly endorse) to 5 (strongly reject):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0": 3, "1": 3, "2": 3, "3": 3, "4": 3, "5": 3, "6": 3, "7": 3, "8": 3, "9": 3}', '{\n "0": 3,\n "1": 3,\n "2": 3,\n "3": 3,\n "4": 3,\n "5": 3,\n "6": 3,\n "7": 3,\n "8": 3,\n "9": 3\n}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.095, 0.095, 0.095, 0.095, 0.095, 0.095, 0.095, 0.095, 0.095, 0.143] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
2026-09-18 20:42:35.007 | INFO | openrouter_wrapper.retry:is_retryable_error:73 - Evaluating error for retry:
|
||||
2026-09-18 20:42:35.007 | WARNING | openrouter_wrapper.retry:is_retryable_error:108 - Retryable: Network/timeout error -
|
||||
2026-09-18 20:49:48.830 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.6-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
Homosexuality
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly endorse) to 5 (strongly reject):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":5,"1":5,"2":4,"3":4,"4":3,"5":3,"6":2,"7":2,"8":1,"9":1}', '{"0": 3, "1": 3, "2": 3, "3": 3, "4": 3, "5": 3, "6": 3, "7": 3, "8": 3, "9": 3}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.078, 0.078, 0.089, 0.089, 0.1, 0.1, 0.111, 0.111, 0.122, 0.122] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
2026-09-18 20:53:50.557 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.7-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
Homosexuality
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly reject) to 5 (strongly endorse):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{\n "0": 1,\n "1": 1,\n "2": 1,\n "3": 1,\n "4": 2,\n "5": 2,\n "6": 3,\n "7": 4,\n "8": 4,\n "9": 5\n}', '{"0": 1, "1": 1, "2": 1, "3": 2, "4": 3, "5": 3, "6": 4, "7": 4, "8": 5, "9": 5}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.04, 0.04, 0.04, 0.052, 0.084, 0.092, 0.124, 0.153, 0.173, 0.201] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
2026-09-18 20:57:35.727 | INFO | openrouter_wrapper.retry:is_retryable_error:73 - Evaluating error for retry:
|
||||
2026-09-18 20:57:35.727 | WARNING | openrouter_wrapper.retry:is_retryable_error:108 - Retryable: Network/timeout error -
|
||||
2026-09-18 21:02:18.642 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.7-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
Homosexuality
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly reject) to 5 (strongly endorse):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0": 1, "1": 1, "2": 1, "3": 2, "4": 2, "5": 3, "6": 3, "7": 4, "8": 4, "9": 5}', '{"0": 1, "1": 1, "2": 1, "3": 2, "4": 3, "5": 3, "6": 4, "7": 4, "8": 5, "9": 5}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.035, 0.035, 0.051, 0.069, 0.097, 0.104, 0.132, 0.138, 0.166, 0.173] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
2026-09-18 21:06:37.223 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.7-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
Homosexuality
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly endorse) to 5 (strongly reject):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":3,"1":3,"2":3,"3":3,"4":3,"5":3,"6":3,"7":3,"8":3,"9":3}', '{"0": 3, "1": 3, "2": 3, "3": 3, "4": 3, "5": 3, "6": 3, "7": 3, "8": 3, "9": 3}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
2026-09-18 21:12:54.301 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.7-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
Homosexuality
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly endorse) to 5 (strongly reject):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":5,"1":4,"2":4,"3":3,"4":3,"5":3,"6":2,"7":2,"8":1,"9":1}', '{"0": 3, "1": 3, "2": 3, "3": 3, "4": 3, "5": 3, "6": 3, "7": 3, "8": 3, "9": 3}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.044, 0.049, 0.072, 0.077, 0.099, 0.099, 0.126, 0.126, 0.154, 0.154] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
2026-09-18 21:16:31.246 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.8-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
Homosexuality
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly reject) to 5 (strongly endorse):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0": 1, "1": 1, "2": 1, "3": 1, "4": 1, "5": 2, "6": 2, "7": 3, "8": 4, "9": 5}', '{"0": 1, "1": 1, "2": 1, "3": 2, "4": 3, "5": 3, "6": 4, "7": 4, "8": 5, "9": 5}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.041, 0.041, 0.047, 0.058, 0.075, 0.091, 0.117, 0.141, 0.182, 0.206] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
2026-09-18 21:25:40.367 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.8-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
Homosexuality
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly reject) to 5 (strongly endorse):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":1,"1":1,"2":2,"3":2,"4":3,"5":3,"6":4,"7":4,"8":5,"9":5}', '{"0": 1, "1": 1, "2": 1, "3": 1, "4": 1, "5": 1, "6": 1, "7": 1, "8": 1, "9": 5}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.041, 0.041, 0.063, 0.069, 0.091, 0.098, 0.12, 0.126, 0.149, 0.203] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
2026-09-18 21:28:53.527 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.8-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
Homosexuality
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly endorse) to 5 (strongly reject):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0": 3, "1": 3, "2": 3, "3": 3, "4": 3, "5": 3, "6": 3, "7": 3, "8": 3, "9": 3}', '{\n "0": 3,\n "1": 3,\n "2": 3,\n "3": 3,\n "4": 3,\n "5": 3,\n "6": 3,\n "7": 3,\n "8": 3,\n "9": 3\n}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.067, 0.067, 0.083, 0.083, 0.1, 0.1, 0.117, 0.117, 0.133, 0.133] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
2026-09-18 21:38:35.148 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.8-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
Homosexuality
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly endorse) to 5 (strongly reject):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0": 5, "1": 5, "2": 4, "3": 4, "4": 3, "5": 3, "6": 2, "7": 2, "8": 1, "9": 1}', '{\n "0": 5,\n "1": 5,\n "2": 4,\n "3": 4,\n "4": 3,\n "5": 3,\n "6": 2,\n "7": 2,\n "8": 1,\n "9": 1\n}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.033, 0.033, 0.067, 0.067, 0.1, 0.1, 0.133, 0.133, 0.167, 0.167] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
@@ -0,0 +1,524 @@
|
||||
2026-09-18 19:54:17.653 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3-flash-preview, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
|
||||
Homosexuality
|
||||
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly reject) to 5 (strongly endorse):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":1,"1":1,"2":1,"3":2,"4":3,"5":3,"6":4,"7":4,"8":4,"9":5}', '{"0":1,"1":1,"2":1,"3":2,"4":3,"5":3,"6":4,"7":4,"8":4,"9":5}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.047, 0.047, 0.047, 0.071, 0.089, 0.095, 0.118, 0.118, 0.13, 0.237] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
|
||||
2026-09-18 20:02:03.874 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3-flash-preview, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
|
||||
Homosexuality
|
||||
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly reject) to 5 (strongly endorse):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":1,"1":1,"2":2,"3":2,"4":3,"5":3,"6":4,"7":4,"8":5,"9":5}', '{"0": 3, "1": 3, "2": 3, "3": 3, "4": 3, "5": 3, "6": 3, "7": 3, "8": 3, "9": 3}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.089, 0.089, 0.094, 0.094, 0.1, 0.1, 0.106, 0.106, 0.111, 0.111] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
|
||||
2026-09-18 20:04:06.878 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3-flash-preview, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
|
||||
Homosexuality
|
||||
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly endorse) to 5 (strongly reject):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":3,"1":3,"2":3,"3":3,"4":3,"5":3,"6":3,"7":3,"8":3,"9":3}', '{"0":5,"1":4,"2":3,"3":2,"4":1,"5":1,"6":1,"7":1,"8":1,"9":1}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.064, 0.073, 0.082, 0.092, 0.101, 0.111, 0.111, 0.122, 0.122, 0.122] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
|
||||
2026-09-18 20:12:24.458 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3-flash-preview, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
|
||||
Homosexuality
|
||||
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly endorse) to 5 (strongly reject):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":5,"1":5,"2":4,"3":4,"4":3,"5":3,"6":2,"7":2,"8":1,"9":1}', '{"0":5,"1":5,"2":4,"3":4,"4":3,"5":3,"6":2,"7":2,"8":1,"9":1}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.078, 0.078, 0.089, 0.089, 0.1, 0.1, 0.111, 0.111, 0.122, 0.122] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
|
||||
2026-09-18 20:14:19.016 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.5-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
|
||||
Homosexuality
|
||||
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly reject) to 5 (strongly endorse):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":1,"1":1,"2":1,"3":1,"4":1,"5":1,"6":1,"7":1,"8":1,"9":5}', '{"0":1,"1":1,"2":1,"3":1,"4":1,"5":1,"6":1,"7":1,"8":1,"9":5}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.054, 0.054, 0.054, 0.071, 0.071, 0.089, 0.107, 0.107, 0.125, 0.268] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
|
||||
2026-09-18 20:19:35.359 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.5-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
|
||||
Homosexuality
|
||||
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly reject) to 5 (strongly endorse):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0": 3, "1": 3, "2": 3, "3": 3, "4": 3, "5": 3, "6": 3, "7": 3, "8": 3, "9": 3}', '{"0":3,"1":3,"2":3,"3":3,"4":3,"5":3,"6":3,"7":3,"8":3,"9":3}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
|
||||
2026-09-18 20:21:30.844 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.5-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
|
||||
Homosexuality
|
||||
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly endorse) to 5 (strongly reject):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":3,"1":3,"2":3,"3":3,"4":3,"5":3,"6":3,"7":3,"8":3,"9":3}', '{\n "0": 5,\n "1": 5,\n "2": 5,\n "3": 5,\n "4": 5,\n "5": 5,\n "6": 5,\n "7": 5,\n "8": 5,\n "9": 1\n}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.09, 0.09, 0.09, 0.09, 0.09, 0.09, 0.09, 0.09, 0.09, 0.186] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
|
||||
2026-09-18 20:26:34.680 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.5-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
|
||||
Homosexuality
|
||||
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly endorse) to 5 (strongly reject):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{\n "0": 5,\n "1": 5,\n "2": 4,\n "3": 4,\n "4": 3,\n "5": 3,\n "6": 2,\n "7": 2,\n "8": 1,\n "9": 1\n}', '{"0":3,"1":3,"2":3,"3":3,"4":3,"5":3,"6":3,"7":3,"8":3,"9":3}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.089, 0.089, 0.094, 0.094, 0.1, 0.1, 0.106, 0.106, 0.111, 0.111] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
|
||||
2026-09-18 20:28:52.548 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.6-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
|
||||
Homosexuality
|
||||
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly reject) to 5 (strongly endorse):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":1,"1":1,"2":1,"3":1,"4":1,"5":1,"6":1,"7":1,"8":1,"9":5}', '{\n "0": 1,\n "1": 1,\n "2": 1,\n "3": 1,\n "4": 1,\n "5": 1,\n "6": 1,\n "7": 1,\n "8": 1,\n "9": 5\n}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.086, 0.086, 0.086, 0.086, 0.086, 0.086, 0.086, 0.086, 0.086, 0.229] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
|
||||
2026-09-18 20:37:45.867 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.6-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
|
||||
Homosexuality
|
||||
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly reject) to 5 (strongly endorse):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0": 1, "1": 1, "2": 1, "3": 2, "4": 2, "5": 3, "6": 3, "7": 4, "8": 4, "9": 5}', '{"0": 1, "1": 1, "2": 1, "3": 2, "4": 2, "5": 3, "6": 3, "7": 4, "8": 4, "9": 5}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.047, 0.047, 0.058, 0.077, 0.088, 0.108, 0.119, 0.138, 0.149, 0.168] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
|
||||
2026-09-18 20:40:18.835 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.6-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
|
||||
Homosexuality
|
||||
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly endorse) to 5 (strongly reject):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0": 3, "1": 3, "2": 3, "3": 3, "4": 3, "5": 3, "6": 3, "7": 3, "8": 3, "9": 3}', '{\n "0": 3,\n "1": 3,\n "2": 3,\n "3": 3,\n "4": 3,\n "5": 3,\n "6": 3,\n "7": 3,\n "8": 3,\n "9": 3\n}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.095, 0.095, 0.095, 0.095, 0.095, 0.095, 0.095, 0.095, 0.095, 0.143] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
|
||||
2026-09-18 20:42:35.007 | INFO | openrouter_wrapper.retry:is_retryable_error:73 - Evaluating error for retry:
|
||||
2026-09-18 20:42:35.007 | WARNING | openrouter_wrapper.retry:is_retryable_error:108 - Retryable: Network/timeout error -
|
||||
2026-09-18 20:49:48.830 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.6-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
|
||||
Homosexuality
|
||||
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly endorse) to 5 (strongly reject):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":5,"1":5,"2":4,"3":4,"4":3,"5":3,"6":2,"7":2,"8":1,"9":1}', '{"0": 3, "1": 3, "2": 3, "3": 3, "4": 3, "5": 3, "6": 3, "7": 3, "8": 3, "9": 3}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.078, 0.078, 0.089, 0.089, 0.1, 0.1, 0.111, 0.111, 0.122, 0.122] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
|
||||
2026-09-18 20:53:50.557 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.7-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
|
||||
Homosexuality
|
||||
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly reject) to 5 (strongly endorse):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{\n "0": 1,\n "1": 1,\n "2": 1,\n "3": 1,\n "4": 2,\n "5": 2,\n "6": 3,\n "7": 4,\n "8": 4,\n "9": 5\n}', '{"0": 1, "1": 1, "2": 1, "3": 2, "4": 3, "5": 3, "6": 4, "7": 4, "8": 5, "9": 5}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.04, 0.04, 0.04, 0.052, 0.084, 0.092, 0.124, 0.153, 0.173, 0.201] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
|
||||
2026-09-18 20:57:35.727 | INFO | openrouter_wrapper.retry:is_retryable_error:73 - Evaluating error for retry:
|
||||
2026-09-18 20:57:35.727 | WARNING | openrouter_wrapper.retry:is_retryable_error:108 - Retryable: Network/timeout error -
|
||||
2026-09-18 21:02:18.642 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.7-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
|
||||
Homosexuality
|
||||
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly reject) to 5 (strongly endorse):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0": 1, "1": 1, "2": 1, "3": 2, "4": 2, "5": 3, "6": 3, "7": 4, "8": 4, "9": 5}', '{"0": 1, "1": 1, "2": 1, "3": 2, "4": 3, "5": 3, "6": 4, "7": 4, "8": 5, "9": 5}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.035, 0.035, 0.051, 0.069, 0.097, 0.104, 0.132, 0.138, 0.166, 0.173] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
|
||||
2026-09-18 21:06:37.223 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.7-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
|
||||
Homosexuality
|
||||
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly endorse) to 5 (strongly reject):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":3,"1":3,"2":3,"3":3,"4":3,"5":3,"6":3,"7":3,"8":3,"9":3}', '{"0": 3, "1": 3, "2": 3, "3": 3, "4": 3, "5": 3, "6": 3, "7": 3, "8": 3, "9": 3}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
|
||||
2026-09-18 21:12:54.301 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.7-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
|
||||
Homosexuality
|
||||
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly endorse) to 5 (strongly reject):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":5,"1":4,"2":4,"3":3,"4":3,"5":3,"6":2,"7":2,"8":1,"9":1}', '{"0": 3, "1": 3, "2": 3, "3": 3, "4": 3, "5": 3, "6": 3, "7": 3, "8": 3, "9": 3}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.044, 0.049, 0.072, 0.077, 0.099, 0.099, 0.126, 0.126, 0.154, 0.154] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
|
||||
2026-09-18 21:16:31.246 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.8-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
|
||||
Homosexuality
|
||||
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly reject) to 5 (strongly endorse):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0": 1, "1": 1, "2": 1, "3": 1, "4": 1, "5": 2, "6": 2, "7": 3, "8": 4, "9": 5}', '{"0": 1, "1": 1, "2": 1, "3": 2, "4": 3, "5": 3, "6": 4, "7": 4, "8": 5, "9": 5}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.041, 0.041, 0.047, 0.058, 0.075, 0.091, 0.117, 0.141, 0.182, 0.206] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
|
||||
2026-09-18 21:25:40.367 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.8-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
|
||||
Homosexuality
|
||||
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly reject) to 5 (strongly endorse):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0":1,"1":1,"2":2,"3":2,"4":3,"5":3,"6":4,"7":4,"8":5,"9":5}', '{"0": 1, "1": 1, "2": 1, "3": 1, "4": 1, "5": 1, "6": 1, "7": 1, "8": 1, "9": 5}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.041, 0.041, 0.063, 0.069, 0.091, 0.098, 0.12, 0.126, 0.149, 0.203] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
|
||||
2026-09-18 21:28:53.527 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.8-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
|
||||
Homosexuality
|
||||
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly endorse) to 5 (strongly reject):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0": 3, "1": 3, "2": 3, "3": 3, "4": 3, "5": 3, "6": 3, "7": 3, "8": 3, "9": 3}', '{\n "0": 3,\n "1": 3,\n "2": 3,\n "3": 3,\n "4": 3,\n "5": 3,\n "6": 3,\n "7": 3,\n "8": 3,\n "9": 3\n}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.067, 0.067, 0.083, 0.083, 0.1, 0.1, 0.117, 0.117, 0.133, 0.133] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
|
||||
2026-09-18 21:38:35.148 | DEBUG | moralmaps.read_api:read_items_rated:487 -
|
||||
=== TRACE read_items_rated first item (google/gemini-3.8-flash, N=6) ===
|
||||
--- prompt ---
|
||||
Please tell me for each of the following statements whether you think it can always be justified, never be justified, or something in between, using this card.
|
||||
|
||||
Homosexuality
|
||||
|
||||
Rate how strongly you personally agree with or endorse EACH of the following answers, on a scale of 1 (strongly endorse) to 5 (strongly reject):
|
||||
0) Never justifiable
|
||||
1) 2
|
||||
2) 3
|
||||
3) 4
|
||||
4) 5
|
||||
5) 6
|
||||
6) 7
|
||||
7) 8
|
||||
8) 9
|
||||
9) Always justifiable
|
||||
|
||||
Respond with ONLY a JSON object mapping each answer's number to its 1-5 rating, e.g. {"0": 2, "1": 5}. Rate every answer, and output nothing but the JSON.
|
||||
--- first 2 raw replies ---
|
||||
['{"0": 5, "1": 5, "2": 4, "3": 4, "4": 3, "5": 3, "6": 2, "7": 2, "8": 1, "9": 1}', '{\n "0": 5,\n "1": 5,\n "2": 4,\n "3": 4,\n "4": 3,\n "5": 3,\n "6": 2,\n "7": 2,\n "8": 1,\n "9": 1\n}']
|
||||
--- mean p over ['Never justifiable', '2', '3', '4', '5', '6', '7', '8', '9', 'Always justifiable'] ---
|
||||
[0.033, 0.033, 0.067, 0.067, 0.1, 0.1, 0.133, 0.133, 0.167, 0.167] valid=6/6
|
||||
SHOULD: replies are a bare JSON dict of 1-5 ratings; valid rate near 1.0 -> coherent. ELSE the record shows malformed output, rescue, or request failure.
|
||||
|
||||
Reference in new issue
Block a user