This document contains instructions on how to download the December 2020 version of the Web Data Commons RDFa, Microdata, Embedded JSONLD, and Microformats data sets.
The extracted RDFa, Microdata, Microformat and Embedded JSONLD data is provided for download as N-Quads. Files are compressed using GZIP and each file is around 100 MB large, except the RDFa files which are around 30MB large. Overall 21,346 files with a total size of 1.9 TB are provided.
The extracted RDF data can be downloaded using wget with the command
wget -i http://webdatacommons.org/structureddata/2020-12/files/file.list. The files, containing quads for a specific formats can be found in the table below, together with more detailed statistics about the number of files and sizes.
If you want to download the class-specific schema.org datasets please navigate to the WDC class-specific subsets page.
|Format||Number Of Files||Approx. Total File Size||File List|
The source code can be checked out from our Subversion repository. The extraction of December 2020 was done with version 1.0.5 of the extractor. For more information about the framework and a detailed description how to run a own extraction visit the framework page.
The analysis of the quads was done using the Version 0.0.1 of the StructuredDataProfiler, which is also available within the Subversion repository.