IDL Data on AWS

The Industry Documents Library (IDL) the UCSF Industry Documents Library (IDL) is a publicly accessible digital archive containing millions of internal documents from industries that impact public health, including tobacco, opioids, chemical, drug, food, and fossil fuel. IDL is hosted at the University of California, San Francisco.
The IDL was established in 2002 as the Legacy Tobacco Documents Library with support from the American Legacy Foundation, now Truth Initiative. IDL partners with Johns Hopkins University to create the UCSF-JHU Opioid Industry Documents Archive.

The IDL Data on AWS contains the metadata, documents, and extracted text for all of the documents, allowing anyone to apply computational methods to these documents.

These documents are primarily internal corporate records publicly disclosed from ongoing litigation brought by local and state governments and tribal communities against manufacturers, wholesalers, distributors, and pharmacies. Document types and topics include:

• Email communications between sales representatives, physicians and other healthcare representatives
• Internal sales training materials, sales rep data, and compensation strategies
• Submissions to regulatory agencies regarding consumer guides, brochures, and prescribing information
• Graphics designs for product packaging and labeling
• Brochures and prescribing publications intended for physicians and the general public
• Advertisements and marketing materials
• Correspondence with physicians
• Other internal documents

Read more about IDL.

Data is available at: [arn:aws:s3:::ucsf-idl-dataset]

Copyright and Fair Use

The University of California, San Francisco have been authorized to serve as the public document repository for documents disclosed in certain litigation according to court orders and to agreements with various state attorneys general. However, ownership of the copyright in these documents has not changed as a result of these court orders and agreements, so copyright in a document is still generally held by the company whose employee(s) created it. Read more on copyright and fair use for these documents.

Attribution

When citing the IDL Data on AWS, please include a link to https://registry.opendata.aws/idl.

Description

The IDL Data on AWS contains a growing collection of over 25 million documents with metadata. New documents are added approximately once a month.

IDLData on AWS includes the following directories and files:

• README.md - The present file.

• f/, g/, h/, j/, etc. - single-character directories containing further single-character directories containing the documents themselves, plus extracted text

• metadata/ - metadata describing the documents

Need Help?

Please contact us at industrydocuments@ucsf.edu with suggestions on how to clarify this documentation and make the IDL Data on AWS more useful!