The ClueWeb22 Dataset: Obtaining a Copy

The ClueWeb22 datasets are distributed by Carnegie Mellon University for research purposes only. A dataset may be obtained by signing a data license agreement with Carnegie Mellon University, and paying a fee that covers the cost of distributing the dataset.

It takes 2-3 weeks to obtain a dataset. Please allow enough time. We do not have the power to hurry the university administrators that must approve your license.

The steps are as follows.

  1. Complete the Organization Agreement and Order Form.

    The Organization Agreement must be signed by a person with the authority to sign agreements on behalf of your organization. The person signing must also initial each page of the agreement on the bottom right corner.

    The Organization Agreement typically applies to a single research group or unit within a larger legal entity. For example, in a university, the Organization Agreement might apply to a research group consisting of a few professors, and the students and staff doing research with them. In this case, the organization would be the name of the research group (e.g., the Information Retrieval Laboratory), and the Corporation/Legal Entity would be the name of the university.

  2. Email the completed Agreement and Order form (all six pages) to Jess Maguire at the Language Technologies Institute (jlm4 at andrew dot cmu dot edu). If you cannot send a pdf, please contact Jess by email to arrange an alternate method.

  3. Jess will send you an email confirmation that we have received your order.

  4. Carnegie Mellon's Sponsored Projects office will send you an invoice for payment. The cost of the various versions of the dataset are shown below.

    Document
    Categories     		Document
    Count     		Document
    Formats
    Distribution media
    Cost*
    B, A, L varies html, jpg, vdom, txt Dataset license only $0
    B 200M txt 1 ✕ 1 TB disk $305
    B 200M html, txt, vdom, in/out links 1 ✕ 14 TB disk $625
    A 2B html, txt, vdom, in/out links 8 ✕ 18 TB disk $4,460
    L 10B txt 2 ✕ 14 TB disk $1,010
    L 10B txt, in/out links 2 ✕ 18 TB disk $1,095
             * Does not include shipping costs

  5. Payment must be in U.S. dollars.

    Note: We are not automatically notified when funds are deposited to CMU's bank account. After you make your payment, please notify Jess Maguire by email (jlm4 at andrew dot cmu dot edu) so that we know to watch for it.

  6. After payment is received, the dataset is shipped to you.

  7. Each individual who will use or have access to the dataset must sign an Individual Agreement. You must retain these signed individual agreements within your organization.