I used the r/ssc CW-thread archive I’d created for my previous analysis to fine-tune GPT-2-345M (with code from nshepperd and very helpful guidance from the tutorial written by u/gwern).
This is similar to the post by u/ratroj a few weeks ago, except mine is trained on the entire history rather than singling out a few controversial comments.
Methodology/Training
For the fine-tuning training set, I included the following metadata for each comment:
The comment’s beginning and end
Whether it was a top-level comment or reply. As I described in my other post, top-level comments were very distinct from other replies in terms of length and style/content, so I thought it was worth differentiating them in training.
The comment ID (e.g. this had an id of “ebgzm5r”) and the ID of its parent comment (if it has one). This was included as an attempt to teach the model the nesting pattern of the thread, which otherwise it would have no information about. My idea was to place the ID at the end of each comment and then to include the parent_id at the beginning, so even with a small lookback window it could hopefully recognize that when the two ids match, the second comment is a reply to the first.
The commenter account name. I included this for training, but I ended up removing it from the example outputs here because it seemed ethically iffy to attribute fake comments to specific real users (especially since some of them have since deleted their accounts).
As a side note, in my experimenting I was impressed with how the trained model correctly learned some of the stylistic/content traits of specific users. For example, in my other post I’d created a list of the top 100 (by volume) commenters sorted by their average comment length. If I prompt the model to write replies using a username from the top of the list (ie someone who usually writes very long comments) the average generated comment is indeed much longer than if I prompt using someone from the bottom of the list. Subjectively, I also think the model did a good job capturing the style / word choice of some of the most-frequent commenters.
I then put all the comments in a txt file in an order mimicking reddit’s “sort by new”, and fine-tuned using that (in hindsight, I realized the results probably would have been slightly better if I’d done reddit’s “top” sort instead).
Once I had the model trained, my method for actually generating the example thread was:
Generate 100 top-level comments by prompting with my “top-level” metadata header.
For each top-level comment, generate replies by appending the parent comment with the header for a reply (correctly matching the parent id).
Similarly, generate replies to the replies by prompting with the “context” (ie the parent and grandparent comments) appended with the header for a reply. Note that I could have done more levels of replies, but the generated text got less coherent as it got deeper, and it occasionally started to return incorrectly-formatted metadata as well.
Results
Anyway, here are the results after around 20,000 steps of training, here after 40,000, and here after 70,000.
Overall, I think the top-level comments were definitely more coherent in the 40K and 70K versions than the 20K, and had fewer formatting errors. For the replies, it was harder for me to tell but it seemed like the 20K version was very slightly better / less overfit. My guess for an explanation is that the replies would be more vulnerable to overfitting since they’re generated using much longer prompts than the top-levels are.
My personal favorite generated comment was this one:
This is from the New Yorker. A former employee of Donald Trump's presidential campaign met a grisly end Friday when he was caught furtively telling his fellow campaign staffers to kiss his butt in a hotel room in August while he was in India.
His co-campaign manager has resigned; his campaign has been running on the principle that it has no tolerance for this behavior. The FBI says it is looking at whether he was also a spy for Russia or is just a disgruntled republican fundraiser.
Want to add to the discussion?
Post a comment!