Text File Upload

The app can analyze any textual data. A corpus should be uploaded as a single CSV file with the following characteristics:

• Has no header.
• Is fully encoded in utf-8.
• Text has to be in the first column, and dates (yyyy-mm-dd) in the second column. No other date formats are accepted.
• Text must have no quoting, and the CSV separator must not be in the text field.
• An optional third column can contain either source weights (numeric values; higher scores indicate more important sources) or a label in the format LABEL_PERIOD or LABEL_PERIOD_CATEGORY, where PERIOD is an integer and LABEL/CATEGORY are strings. The format must be consistent across all rows.
• Each user has a maximum allowed file size. Uploading bigger files is not allowed: they will be automatically deleted after upload.

You can analyze only one file at a time. The uploader will retain only the last uploaded CSV file, deleting the others.

Validate Text File

This function is used to validate the uploaded CSV file and verify that there are no errors. Please:

• specify the CSV separator used in the CSV file (use a single character with no quoting) and the minimum length of documents you want to consider (they need to have at least three characters). Maximum document length is 2,000,000 characters;
• optionally choose to validate the third column using the Validate third column dropdown: select Source weights to verify that the third column contains numeric values, or LABEL_PERIOD or LABEL_PERIOD_CATEGORY to verify that it follows one of those two label formats consistently across all rows. Select Do not validate to skip this check;
• optionally choose to automatically remove lines with errors. Please be careful, as this will change the uploaded file;
• optionally choose to remove or keep all the text documents (file rows) that match a specific search query. This will significantly slow down the validation process. The following parameters can be used:
  • Use search operators: this option can be used to remove or keep only the documents that match the search query specified through the following fields. If the “No” option is selected, the system will only identify and list the documents matching the query without keeping or removing them. A preliminary run with the “No” option selected is recommended to see which (and how many) documents are matched by the query.
  • Doc includes ALL these words: matched documents will include ALL the words listed here. List words with no quoting or spacing, only separated by a comma. Use lowercase and only letters (no punctuation).
  • Doc includes AT LEAST ONE of these words: matched documents will include AT LEAST ONE of the words listed here. List words with no quoting or spacing, only separated by a comma. Use lowercase and only letters (no punctuation).
  • Doc DOES NOT INCLUDE ANY of these words: matched documents will NOT include any of the words listed here. List words with no quoting or spacing, only separated by a comma. Use lowercase and only letters (no punctuation).

Output

The validated (and corrected) file will be automatically saved on the system (replacing the original CSV). There is no need to reupload a correctly validated file. The system will also offer the possibility to download the input file that has been corrected.

Data Management

The "Delete All My Data" button allows users to permanently erase all user data and saved reports from the system. Please use this feature with caution, as the deletion is irreversible and cannot be undone.