Synchronizing Banner and EDIR LDAP Data

# 20070806 sxelm Verifying Directory/Registry are in Synch

The production EDIR directory instance, slapd-<server>Prod, is associated with an Oracle registry instance, OPS$SXLDAP in RPTS. Monday through Friday, both the production registry and one instance the production directory are dumped to text files that can be compared to determine if the directory and registry remain in synch.

The dump is automatic (scheduled in cron) and normally completed by 8:30am.

summit:~sxldap/local/ldap/scripts/dump_all.ksh

The comparison is also automatic (scheduled as cron) and starts a 9am.

summit:~sxldap/local/ldap/scripts/compare_dumps.ksh

The compare job generates the email with the subject 'EDIR: out of sync (RPTS)' and also generates a number of files that can be used when resolving out of sync conditions.

What is not automated is the process of generating LDIF to put the registry and directory back in sync. When the problem is solely related to Banner extracted data, the following resolution can be applied.

#================================================================================
# Resolution when only Banner extracted data out of sync
#================================================================================

# Use ~sxldap/local/ldap/cleanup/problems.update_last_ldif to modify registry # records before rerunning the ldap_gen_ldif procedure # to generate LDIF for updating the directory.

ssh -l sxldap localhost cd local/ldap/cleanup/

# Establish RPTS environment . ua_oracle RPTS env

# Connect to registry sqlplus /

# Update registry table @$PWD/problemsRPTS.update_last_ldif

# !!!!!!!!!!!!! # After flagging the problem accounts for ldif generation, # you can simply wait for the ldif to be generated and applied # the following working day. # # Alternatively, you can generate and apply the ldif the same day # with the following steps - but you will need DBA support for # one step. # !!!!!!!!!!!!!

# Generate LDIF @../registry/execute_xprocess

# when prompted for input, enter the following exactly ldap_gen_ldif('both','all',return_status)

# This will generated LDIF for the accounts that are # out of synch between the directory and the registry.

#+++++++++++++++++++++++++++++++++++++
# As oracle, execute the following scripts to remove # /tmp/ldap_RPTS output already copied to the sxldap # account and to change permissions on those files # that were just generated so that sxldap can copy them # to his own directory.

# connect to oracle account on summit ssh -l oracle summit.alaska.edu

# Run cleanup script to remove previously processed # ldap_RPTS files that remain in /tmp. Because the # files recently created have not been processed, # PMldap_tmp_cleanup.ksh will report errors. That's OK. $HOME/local/production/PMldap_tmp_cleanup.ksh

# Run chmod script to make it possible for sxldap to # copy the newly generated /tmp/ldap_RPTS files: $HOME/local/production/PMldap_tmp_chmod.ksh #+++++++++++++++++++++++++++++++++++++

# As sxldap, copy the /tmp files to $HOME/local/ldap/extracts # and place copies on eklutna and edgar. $HOME/appworx/manage_ldif_files.ksh

# Then process the LDIF on eklutna and edgar. $HOME/appworx/apply_ldif_files.ksh

#================================================================================
# Commentary
#================================================================================

The directory and registry are expected to be in synch at all times. Since the dump files are point in time copies made from different sources on different hosts, however, it is possible for discrepancies to be reported that are in fact only timing differences. Gross discrepancies are almost always associated with dumps taken at distinctly different times or dumps being performed while daily batch updates are still being processed.

To determine if gross discrepancies are related to timing differences, compare dump file date stamps with log file date stamp in:

eklutna:~iplanet/local/spool/ summit:~sxldap/local/ldap/extracts/

Discrepencies that are significant, and require resolution, are those associated with attributes extracted from Banner (not self service, therefore only the extract process touches them) and self service attribute differences not associated with an update transaction.

The Banner extract, aggregation and LDIF generation processes for EDIR are pretty clean as of 2004/10/20. However, we do still encounter circumstances not allowed for by the EDIR batch processes, circumstances that results in registry changes not being propogated to the directory. The resynching steps (above) propogate the registry changes to the directory but do not resolve the undering processing problem that caused the discrepancy.

The troubleshooting and debugging of the EDIR Banner extract process is beyond the scope of this document.

Self service updates, processed by the EDIR web gateway script ldap_bulk_update, are very clean as of 2004/10/20. Because the gateway uses a two phase commit for directory updates (update must pass registry constraints or is not processed in directory; if update fails directory constraints registry update is rolled back) it is hightly unlikely that a self service update will fail to be recorded in the registry. Should that ever occur, the resolution is to insert a corresponding record into OPS$SXLDAP.LDAP_ATTR_SS. Self service updates are recorded by EDIR in date stamped gateway_edir_updates files in eklutna:~iplanet/local/ldap/web/log/. Those files are renamed with a "COPIED." prefix when transfered to the backup directory on edgar.

The troublshooting and debugging of EDIR web gateway ldap_bulk_update script is beyond the scope of this document.

(eof)