Automate your File based Data Loads in FDMEE with Open Batches
In one of my recent implementation assignments, we were faced with the requirement of eliminating any manual intervention and automating our data loads in Financial Data Management Enterprise Edition (FDMEE), the target system being Hyperion Financial Management (HFM) and Hyperion Tax Provisioning (HTP). Open Batches in FDMEE is an effective way of automating your file based data loads. Though the standard User Guides contain precious little on creating and running Open Batches and are not entirely comprehensive, Open Batches are easy to use and implement. In this blog, I attempt to explain in detail the process of creating Open Batches. These details have come purely from my personal research and my experience from my past assignments.
What is an Open Batch?
Batches play a critical role in automating the Data/Metadata load process. By creating batches, we can easily club different rules together or execute different rules in an automated way by leveraging the scheduling capabilities of FDMEE.
An Open Batch is instrumental in importing data into a POV in FDMEE from a file based data source. Open Batches can be automated to run at scheduled timings. Most importantly, Open Batches pitch in when we have to load more than 1 data file for any particular location. Also, Open Batches provide the flexibility of loading these multiple data files in 'Serial' or 'Parallel' mode.
Configuring and Executing an Open Batch
Let us assume that we have 2 FDMEE Locations and from each of these locations, we have to load 2 data files:
Step 1: Setting up the required FDMEE Locations
We create 2 locations as shown below (without deep-diving into the details of an FDMEE Location)
a. LOC_Sample_Basic1
b. LOC_Sample_Basic2
Step 2: Setting up the Open Batches
A new Open Batch may be created from: Setup -> Batch -> Batch Definition. Click on 'Add' to add a new Batch Definition.
Key in the basic details such as NAME (Name of the Open Batch), TARGET APPLICATION, WAIT FOR COMPLETION and DESCRIPTION. Apart from these, there are a few other details as below:
Type: Select 'Open Batch'.
Execution Mode: Select 'Serial' if you wish to load all the data files in that location one by one. Select 'Parallel' for loading all the files simultaneously.
Open Batch Directory: This is where the User Guides are specifically vague. All we need to do is to specify the name of the folder where the data files will be placed. But, we need to take special care that this folder is created at a level which is lower than the level of the folder 'openbatch'. So, assuming that we need to place the data files in a folder 'Test1', this folder will be created as shown below:
File Name Separator: For File Name Separator, we have to choose from (~ , _ , @ or : ).
Naming convention of the Data Files: For a data file to be executed from an Open Batch, it needs to have the following components in it name, separated by the chosen 'File Name Separator':
File ID: free-form value used to sort the files for a given Location in an order. The values for this can be 1,2,3... or a,b,c... etc, just to specify the order in which the files will be loaded.
Data Load Rule Name: The name of the Rule through which the file will be loaded
Period: The period for which the file is to be loaded
Load Method: 2-character value where first character denotes 'Import Mode' (possible values 'A' and 'R' for 'append' and 'replace' respectively). The second character denotes 'Export Mode' (possible values 'A', 'R', 'M' and 'S' for 'accumulate', 'replace', 'merge' and 'replace by security' respectively).
So, now let us assume that we have to load 2 data files from the location 'LOC_Sample_Basic' using the Data Load Rule 'DLR_Sample_Basic'. Assuming that the 'File Name Separator' chosen is '@', the file names for the 2 files will be:
1@DLR_Sample_Basic@Jan-17@RR
2@DLR_Sample_Basic@Jan-17@RR
The 2 Open Batches for loading the data files will be as below:
OP_Sample_Basic1
OP_Sample_Basic2
Step 3: Configuring the Master Batch
After doing all the above configurations, the default question would be that 'Where is the automation in this when we have ended up creating 2 or more Open Batches? How will we execute 2 or more Open Batches without Manual Intervention?'. The answer to this query lies with the creation of the open batch type 'Batch'. We configure an open batch with the type as 'Batch' and name it 'BatchMaster'.
The feature of the Batch is that we can add multiple Open Batches to the Batch Master and then, schedule just this one batch instead of worrying about multiple Open Batches. To add the 2 Open Batches that we had created earlier to this BatchMaster, click the 'Add' button and enter the details. Ensure that you enter the Job Sequence number to tell the system which Open Batch to execute first.
Step 4: Executing the Open Batches
The Open batches or the BatchMaster can be executed from: Workflow -> Other -> Batch Execution
You may simply select the BatchMater now and schedule it easily through the 'Schedule' button.
FDMEE Multi-Period Data Load
There may be a requirement to load data files that contain data/amount values for multiple periods. Files for an open batch multiple period load are stored in 'inbox\batches\openbatchml' directory. There are just a few minor variations between setting up a Batch for Multi-Period Load and a Single Period Load.
Open Batch File Name: The file name for multi-period files should be as per the format 'FileID_RuleName_StartPeriod_EndPeriod_Loadmethod':
FileID: free-form value used to define the load order
RuleName: Name of the data load rule which will be used to process the file
StartPeriod: First period in the data file
EndPeriod: Last period in the data file
Load Method: Import and Export Mode
So, a multi-period file name would look like for a Data Load Rule named 'Actuaload':
1_Actuaload_Jan-17_Dec-17_RR
2_Actuaload_Jan-17_Dec-17_AR
Open Batch Directory: We specify the name of the folder in the same way as we did for single period batch. But, we need to take special care that this folder is created at a level which is lower than the level of the folder 'openbatchml' instead of 'openbatch'. So, assuming that we need to place the data files in a folder 'Test3', this folder will be created as shown below:
Sample Open Batch for Multi-Period:
