Structure of ctxload Text Load File

The ctxload text load file must use the following format for each document, as well as any structured data associated with the document:

<TEXTSTART: col_name1=doc_data, col_name2=doc_data,...col_nameN=doc_data>
text. . . 
<TEXTEND>

where:

<TEXTSTART: ... >

is a header marker that indicates the beginning of a document. It also may contain one or more of the following fields used to specify structured data for a document:

col_name

is the name of a column that will store structured data for the document.

doc_data

is the structured data that will be stored in the column specified in col_name.

text

is the text of the document to be loaded or the name (and location, if necessary) of an operating system file containing the text to be loaded.

Note: The data in text (either a string of text or a file name pointer) is treated by ctxload as literal data, including any non-alphanumeric characters or blank spaces that may occur. As a result, you must ensure that text exactly represents the data you wish ctxload to process. For example, if you use ctxload to load text from separate files, the file names in the load file must exactly represent the name(s) of the operating-system file(s) containing the text. If any blank spaces are included in a file name, ctxload cannot locate the file and the text is not loaded.

<TEXTEND>

indicates the end of the document.

Load File Structure

The following conditions apply to the structure of the load file:

• for each document to be loaded, either the text of the document or a pointer to a separate file must be in the load file.

• embedded text and separate file pointers cannot be used together in the same load file

• if the text for your documents is embedded in the load file, the text must be in ASCII format

• if pointers to separate files are used, the text in the files can be in plain (ASCII) format or a proprietary format (e.g. MS Word)

• if the text in a separate file is in a proprietary format, the format must be supported by ConText and it must be loaded into a LONG RAW column

• each separate file must contain a single document (the contents of a separate file are stored as a single row in the table)

Load File Syntax

The following conditions apply to the syntax utilized in the text load file:

• <TEXTSTART:...> and <TEXTEND> must each start on a new line

• the structured data parameters within the <TEXTSTART:...> string do not have to be in any particular order

• a newline character (either hard or soft return) cannot occur between a col_name and the beginning of its associated doc_data

  Note: The entire value for doc_data does not have to be on the same line as the col_name; only the beginning of the value and the col_name must share the same line.

• the first col_name should be on the same line as the 'TEXTSTART:'

• the '>' character which indicates the end of the <TEXTSTART:...> string must be on the same line as the last doc_data field for the document

• structured and LONG data may span more than one line

• single quote-marks must be escaped in doc_data (e.g. don't must be entered as don''t)

• each <TEXTSTART: ... > string must be followed by the text of a document or a pointer to a separate file

• the text or file pointer must be placed after the complete <TEXTSTART: ... > string and must start on a new line

• the last character in the load file must be a newline character

Example of Embedded Text in Load File

The following example illustrates a correctly formatted text load file containing structured employee information, such as employee number (1000, 1024) and name (Joe Smith, Mary Jones), and the text for each document:

<TEXTSTART: EMPNO=1000, ELNAME='Smith', EFNAME='Joe'>
Joe has an interesting resume, includes...cliff-diving.
<TEXTEND>
<TEXTSTART: EMPNO=1024, EFNAME='Mary', ELNAME='Jones'>
Mary has many excellent skills, including...technical,
marketing, and organizational.  Team player.
<TEXTEND>

Example of File Name Pointers in Load File

The following example illustrates a correctly formatted text load file containing structured employee information, such as employee number (1000, 1024) and name (Joe Smith, Mary Jones), and a file name pointer for each document.

<TEXTSTART: EMPNO=1024, EFNAME='Mary', ELNAME='Jones'>
mjones.doc
<TEXTEND>
<TEXTSTART: EMPNO=1000, EFNAME='Joe', EFNAME='Smith'>
jsmith.doc
<TEXTEND>

Note: To use the load file in this example, you would have to specify the -separate argument when executing ctxload.