Adding Hash While Uploading File to Al11

Nigh organizations employing SAP's business warehousing solutions utilize flat files (such as the .CSV format) for uploading a variety of transactional business data into their systems.

Typically, the loaded data is further processed, cleansed and consumed in upstream systems with business organization logic.

The transactional information files are oftentimes prepared by functional business organization users, and human being errors within the files tin lead to erroneous information in the system, causing significant challenges. One type of fault concerns special characters. The mistake is typically in the description fields due to word processing errors or copy-paste operations. The data load process rejects the special characters while processing the rest of the data, causing the information request to appear as erroneous in the persistent staging area (PSA) layer or fail to procedure altogether.

This blog mail explores a solution to address special characters by utilizing a combination of existing toolsets, and a scrap of customization.

The Challenge

SAP Business Warehouse (SAP BW) offers an out-of-the-box solution for treatment special characters using the standard transaction code RSKC. Information technology allows users to maintain all special characters that are immune within SAP BW (e.g. advanced DataStore Objects [DSOs], InfoCubes, etc.) This blog mail service addresses the challenge of processing flat files containing special characters before the data is loaded into the SAP BW organisation.

The main challenge is exploring and identifying a solution that meets the following criteria:

1. Permit business organization transaction data in the flat file that contains special characters to be candy seamlessly into SAP BW layers. All special characters appear as "#" when flat files in AL11 folders are viewed or browsed, as shown in the figure below.
2. Optionally allow special characters to exist replaced with a configurable replacement.

Solution

Fortunately, SAP provides a rich framework of toolsets inside the SAP BW system that tin can exist leveraged to build a robust solution for a seamless user experience. This toolset includes InfoPackages with ABAP routines, and transformations with First/END routines where ABAP code can be deployed to access most objects within the Arrangement (InfoObjects, standard and custom tables, etc.)

Assumptions

• Apartment files are available for processing in an AL11 directory at a defined location.
• This solution has been adult and deployed on the SAP BW organization with the specifications listed beneath, only is also applicable to SAP BW/4HANA systems.

Code Page Setting in InfoPackage

The character encoding is a coded character set for which a unique number has been assigned to each character. Information technology reflects the style the coded character set is mapped to bytes for manipulation in a computer. There are many character encoding sets, for example, UTF-8, UTF-16, and UTF-32. In the SAP world, graphic symbol encoding is named with code page which is a four-digit number. Some examples are listed in the table below. Algorithms associated with the code page are used to interpret incoming datasets in the flat file. The figure below shows the code folio setting options in the InfoPackage, which offers two main options to process apartment files.

The first is to load text-type files from the awarding server. Yous'll need to configure a routine to process the flat files. To do this in SAP BW, refer to the outset effigy below. To practice this in SAP BW/4HANA, refer to the 2nd figure.

Next, specify a flat file name (which requires a separate program for further processing).

Alternatively, you can load text-type files from a local workstation (which likewise requires a separate program for further processing), merely this option does non present any claiming in processing special characters.

InfoPackage Configuration

In club to configure the InfoPackage, follow these steps:

Character Set Settings

1. Default settings: Unicode systems volition use UTF-16LE encoding (lawmaking page = 4103).
2. Direct input: allows you lot to override the system lawmaking page using the "graphic symbol set" selection:
3. System lawmaking page : Default, set up past organization equally 4103 (UTF-16LE Unicode).
4. Frequently used SAP code folio : 1100 (based on ISO 8859-one); covers the characters of Western Europe, North and S America, Australia, and Africa.
5. Other SAP code pages : 1401 (Eastern European languages), 8000 (Japanese), 8400 (simplified Chinese), 8300 (traditional Chinese), 1500 (Russian), and more are likewise based on one ISO 8859 code page.

Replacement Character

When incoming apartment files take a grapheme that cannot exist interpreted using algorithms of the configured code page, the arrangement assigns a replacement character and the default is "#".

Endian Setting

In a binary number consisting of multiple bytes (eastward.g. a 32-scrap unsigned integer value), you lot tin can choose 1 of two options. The first is little endian, when the least-significant byte is encoded first and the remaining bytes are encoded in increasing order of significance. The second is large endian, when the most-significant byte is encoded start and the remaining bytes encoded in decreasing order of significance.

For binary and text manner files, the endian setting is "default," and for legacy mode files, either lilliputian endian or large-endian settings could exist configured.

Apartment File Processing in Infopackage Routine or ABAP Programme

In order to perform operations such equally functional validations, etc., on the application server, the flat file needs to exist processed using the below syntax:

OPEN DATASET <P_FILENAME> FOR INPUT IN <Way> ENCODING DEFAULT IGNORING CONVERSION ERRORS.

      *Perform File Operations such every bit reading, writing etc.

Close DATASET <P_FILENAME>.

Where:

• < P_FILENAME> is the name of the application server file
• <Manner> is one of the about common modes, BINARY or TEXT
• TEXT Fashion: The file is opened to be read or written line-by-line. Final infinite characters are deleted in this style.
• BINARY MODE: The file is opened to be read or written to, without whatever line breaks. The exact content of memory is copied.

Each mode variant can be qualified past IGNORING CONVERSION ERRORS to allow the arrangement to suppress any conversion errors at runtime while reading/writing from/to the apartment file. Whenever a grapheme is replaced by a replacement grapheme while reading or writing, the exception divers in the course CX_SY_CONVERSION_CODEPAGE is raised and the specification IGNORING CONVERSION ERRORS allows information technology to suppress.

Replacing Special Characters in Transformations

The solution described above addresses the challenge of processing special characters through InfoPackages to the PSA layer. These special characters can be processed into subsequent layers, but might cause display bug in other upstream layers (e.g. Bex or SAP Business Planning and Consolidation or SAP Power BI). The adjacent (only optional) challenge is to replace them with business-defined characters.

This mail explores a simpler way of replacing special characters. Follow these steps.

1. Store all valid characters (alphabets, numbers, and commonly used symbols) in a custom table (like the one shown below). This is i way of storing special characters; at that place could be numerous other ways.

2. Run the start routine inside SAP BW transformation
3. Access all valid characters (using keys such as VARIANT, SOURCE, DIMENSION, etc. in this example) and store them in a local variable. The character sets may need to concatenate depending upon how they were stored in the table.
4. Compare the contents of the target clarification (or other fields) against this local variable. The logic could deploy "CO" (Contains Only) ABAP operations or any other character or string operation.
5. Supplant the contents of the target description with some other pre-defined content.

Results

Using the workaround explored above, permit's meet how this affects the data load. We configured an InfoPackage in our example with these criteria:

• Graphic symbol prepare settings: directly input
• System code page: 4103 (overridden past character set)
• Character set (1100)
• Replacement character: #
• Endian setting: default

From there, we processed a flat file containing special characters (equally shown in the first figure nosotros looked at) through the InfoPackage. Subsequently successful processing, the data appears in the PSA layer with special characters shown highlighted.

This data was further successfully loaded into the next layer (i.e. advanced DSO) with special characters shown highlighted.

Conclusion

Organizations employing SAP Business Warehousing or SAP BW/4HANA utilize flat files for uploading a variety of transactional business concern data into their systems. These transactional data sets may incorporate special characters which are rejected when candy through standard data load processes, or even neglect to process altogether. In this mail, nosotros presented a solution to help overcome this upshot.

Thanks to Rajesh Hemnani for his assistance with this postal service.

johnsonpritur.blogspot.com

Source: https://blog.sap-press.com/how-to-handle-special-characters-in-sap-bw4hana-data-loads

Share this post