Append and New File Options
The WriteOption property includes options to append data to an existing file or create a new file.
.gif) Note: |
|---|
|
An append operation requires the metadata of the appended data to match the metadata of the data already in the file.
|
The following table describes the available options for the WriteOption property.
|
Option
|
Description
|
|---|
|
Append
|
Appends data to an existing file. The metadata of the appended data must match the file format.
|
|
Create always
|
Always creates a new file.
|
|
Create once
|
Creates a new file. If the file exists, the component fails.
|
|
Truncate and append
|
Truncates an existing file and then writes the data to the file. The metadata of the appended data must match the file format.
|
Using the Raw File Destination in a Loop
If the data flow that uses the Raw File destination is in a loop, you may want to create the file once and then append data to the file when the loop repeats. To append data to the file, the data that is appended must match the format of the existing file.
To create the file in the first iteration of the loop, and then append rows in the subsequent iterations of the loop, you need to do the following at design time:
-
Set the WriteOption property to CreateOnce or CreateAlways and run one iteration of the loop. The file is created. This ensures that the metadata of appended data and the file matches.
-
Reset the WriteOption property to Append and set the ValidateExternalMetadata property to False.
If you use the TruncateAppend option instead of the Append option, it will truncate rows that were added in any previous iteration, and then append new rows. Using the TruncateAppend option also requires that the data matches the file format.
Using the Format of a Nonexisting Raw File
If you want to use the format of an existing raw file but the file is not available at run time, you can specify the existing raw file at design time and this way get the metadata that the Raw File destination requires. Before you deploy the package, set the ValidateExternalMetadata property of the Raw File destination to false. At run time, a new file that has the correct format is created. In this scenario, the package will fail validation at run time if ValidateExternalMetadata is not set to false.