|
<< Click to Display Table of Contents >> Navigation: Modules > Asset Management > Disposals > Reconciliation |
Disposals Reconciliation is the task of taking all the records currently in the Disposal Staging table and consilidating them into the main repository, namely the Disposal Items table. During this process, several dates are set that allow reconciliation between the various stages of the disposal process. The Staging table is where raw data sourced by the various catch points resides. Disposals Reconciliation takes that data, applies business rules, and merges it into the main disposal tables.
Since reconciling records can be a potentially massive job (thousands of records at a time), the Disposals Reconciliation engine runs as a process on the MIE. The process monitors the MIE Job Queue. When it sees a job in the queue in “SUBMITTED” status, it picks it up, changes the status to “PROCESSING”, and begins to process the job. The service monitors the queue in a very tight loop. So that a processing job that gets kicked off does not take the attention of the whole server (making other jobs wait in the queue until it is done), each Disposals Reconciliation job is run as a separate thread. That way, there can be as many Disposals Reconciliation jobs running in parallel as the server can support, which is based on its horsepower and whatever else it’s being asked to do.
The source is the Disposals staging table, which is a very wide table with many columns (FLAT). The destination is comprised of the MIE’s Asset Management data schema, namely the following tables: ATdataDisposals and ATdataDisposalItems.
There are many rules enforced:
1. MIE table definition rules – the MIE maintains several settings for each table. Those settings include:
i. InsertOnly – records can only be inserted into the table, no updates
ii. Table can not be added to – no records may be added to the table through automated feeds
iii. Log changes to existing records
iv. Blank table first before loading
The consolidation routine honors the InsertOnly flag, the Cannot Add records flag, and the log changes flag. The blank first flag is not honored.
2. Field validations. The consolidation process compares fields in the staging table to the User Interface (UI) data entry rules for the various fields. The Disposal Items screen is used as the source of the validation rules. If the field fails the validation check the record is marked as an exception. Clients can override the default data entry form validation rules and have them apply to their Disposal consolidation process.
The Disposals Reconciliation routine will honor the warning level setting that is set through the SETTINGS menu option. The default is 4 (highest). At warning level 4, there are potentially a large number of warnings as every time a catch point sources insufficient data to uniquely identify a record, a level 4 warning is logged.
For every table that is defined as requiring the logging of change, the Disposals Reconciliation routine will log to the change log every field of every record that is changed. The table, primary key, name of field, old value, and new value data elements are logged. This maintains complete traceability as to how data elements were changed. It also allows a future roll-back capability.
Warnings and errors, along with the number of inserts, updates, field-level changes, and the time (in seconds) that it took to run the job are logged to the job log. The entry in the job queue that kicked off the job is linked to the job log entry. There is potentially quite a bit of information logged to the job log.
When a Disposals Reconciliation job is run, only records in the Disposals staging table that have no RECONCILEDSTATUS value or have a RECONCILEDSTATUS value other than “STAGED” are included. If any errors occur during reconciling the record, the RECONCILEDSTATUS field will be set to “Exception”. Otherwise, the RECONCILEDSTATUS field will be set to “STAGED”.
The Disposals setting DisposalToStagingWhereClause specifies which Disposal Item records are copied to the main MIE asset staging table for Consolidation. For example, the where clause can specify that only records that contain an Asset Number of a certain length should be copied to the main MIE asset staging table for asset consolidation. This allows Disposals Reconciliation of all disposal items, and additional Consolidation of those disposal items that also happen to be tracked assets.