Add SCROLLS long-form generation prompts#818
Add SCROLLS long-form generation prompts#818haileyschoelkopf wants to merge 3 commits intobigscience-workshop:eval-hackathonfrom
Conversation
|
|
||
| {{ output }} | ||
|
|
||
| The following is a concise summary of the document: ||| {{ input }}' |
There was a problem hiding this comment.
I'm still weirded out that you can have trailing "\n"
There was a problem hiding this comment.
Sorry where's the trailing "\n"? Maybe I'm missing something
There was a problem hiding this comment.
sorry *can't I would expect the generation to be "below" the "The following is a concise summary of the document:" instead of to the right.
There was a problem hiding this comment.
Oh I see. Yeah does promptsource allow for that? Not too familiar with the convention in PS / why it can't have a trailing "\n"
There was a problem hiding this comment.
It doesn't allow for it; \n & whitespace is stripped
| jinja: 'The following is a summary taken from a government report. | ||
|
|
||
| Summary: {{ output }} | ||
|
|
||
| The contents of that report were likely as follows: ||| {{ input }}' |
There was a problem hiding this comment.
Not sure what that brings compared to previous prompt
| Summary: {{ output }} | ||
|
|
||
| The query given and the original meeting transcript\ | ||
| are as follows: ||| {{ input }}' |
| jinja: 'Here is a summary of a meeting: | ||
|
|
||
| {{ output }} | ||
|
|
||
| The following is the question used to create the summary,\ | ||
| followed by the meeting transcript itself: ||| {{ input }}' |
There was a problem hiding this comment.
Not really different from the previous one no?
There was a problem hiding this comment.
Can remove these duplicate prompts!
3 participants
sorry for the delay--adding SCROLLS prompts to the dataset, favoring prompts which ask for long-form generation based on the text summary where possible. Still working through with more prompts.
@Muennighoff @thomasw21 @lintangsutawika