For the past week I've been working with some real complicated flat files. So I've came across with some hints that I'm going to share with you.
Hint #1
Let's say that you have the following flat file definition:
| Definition | Appear how many times |
Tag: HEA FileNumber: (length = 4) | 1 |
Tag: MSG MsgID: (length = 3) FreeText: (length = 100) | unbounded |
Tag: EOF FileNumber: (length = 4) | 1 |
One example for this definition could be:
HEA1234 MSG001This is a test EOF1234
|
Using the above example to create our schema using the flat file schema wizard, probably you'll get something like this:
Now try to validate the following flat file:
HEA1234 MSG001This is a test MSG002This is another test EOF1234
|
Probably you're getting this error:
C:\test.flat.txt: error BEC2004: Unexpected data found while looking for: '\r\n'
The current definition being parsed is Root. The stream offset where the error occured is 51. The line number where the error occured is 3. The column where the error occured is 20.
This happens because the second MSG element is larger that the one we used to create the schema. As you can see in our initial definition, the FreeText field has a maximum length of 100 characters. So you have to set the Positional Length field to 100.

Hint #2
But if we validate it again we get the same error but in a different position. The problem is that we specified 100 characters of length and neither of FreeText fields in our example has that length.
The solution lies on a magic field called Allow Early Termination. According to this, the Allow Early Termination field "indicates whether positional records can terminate early or must contain data for all record fields." By default is set to No, which means that the record must contain data for all the fields. To solve our problem we must set it to Yes. Click in the <Schema> node and you can find the Allow Early Termination field.
Now if we validate it once again we get a success message. I hope this helps someone.
Until next time ;)