#1 The model card & the video clearly warned about the limitations and problems raised by the model & the POL section of 4Chan in general In fact, the experiment of having the model post messages on 4chan was IMO pretty bad and inappropriate and if the author would have asked us, we would probably have tried to discourage them from doing it.Īfter a lot of internal debate at HF, we decided not to remove the model that the author uploaded here in the conditions that: Thanks for the messages don't advocate or support the training and experiments done by the author with this model. Hi! Clem, co-founder and CEO at Hugging Face here. The latter even has a requirement for applicants to pass a course in human research ethics prior to obtaining access to the data.Ī similar system should be in place here, and be used as the template for future model sharing where the model has the potential to produce harm. See the NIH/TCIA and MIMIC datasets for examples. It is common to allow research access to datasets in this context via a registration platform, where the applicants who are seeking access must describe their proposed research, and sign an agreement for data use. The medical community has well established guidelines on how to manage the sharing of research materials which involve a risk to human subjects, with data privacy being the most common risk. Given the demonstrated risk of harm, this model should not be freely accessible. There is no question that such human experimentation would never pass an ethics review board, where researchers intentionally expose teenagers to generated harmful content without their consent or knowledge, especially given the known risks of radicalisation on sites like 4chan. The model author has used this model to produce a bot that made tens of thousands of harmful and discriminatory online comments on a publicly accessible forum, a forum that tends to be heavily populated by teenagers no less. It should be clear that this model carries a significant risk for this sort of harm, given the fact such an experiment has already been performed.

Human research ethics is baked into the very foundation of our field, because of a long history of human rights abuses in the name of science, in particular experiments that cause harm to disempowered and marginalised people without their consent.

While open science is a great principle, I'm a medical doctor and safety researcher by training and we always need to consider possible harms. The main concern I have is that this model is freely accessible for use. There is nothing wrong with making a 4chan-based model and testing how it behaves.

You shouldnt share thread archive 4chan