As the two major enterprise linux distributions (SUSE and Red Hat) have decided to remove OpenLDAP from their platforms, there has been and will continue to be interest from major deployments wanting to move from OpenLDAP to 389 Directory Server on SLE and Red Hat Directory Server on RHEL. We should not only provide supporting tooling to make this possible, but also document the possible migrations paths and strategies to give customers a long term, supported, high quality LDAP install.
Note this excludes FreeIPA/RHIDM which has it’s own LDAP migration tooling, and that not all deployments can move to IPA due to their requirements.
This document will serve not only as a technical discussion of the approach, but also as a reference for downstream documentation teams on possible migration approaches.
389 DS and OpenLDAP while both being LDAP servers, are very different projects, with different features, configurations and approaches. This means there is not a 1:1 parity between the two, so there will always be situations or configurations that can not be adapted fully.
| Feature | OpenLDAP | 389 DS | Compatible ? |
|---|---|---|---|
| 2 way Replication | SyncREPL | 389 Specific system | ❌ |
| MemberOf | Overlay | Plugin | ✅ (simple configs only) |
| External Auth | SASL Authd | PTA Plugin | 🔨 (Config and Data Changes needed) |
| LDAP Proxy | Proxy | - | ❌ |
| AD Synchronisation | - | Winsync Plugin | ❌ |
| Inbuilt Schema | OLDAP Schemas | 389 Schemas | ✅ (Complete, some esoteric types not supported) |
| Custom Schema | OLDAP Schemas | 389 Schemas | ✅ (Complete as part of openldap2ds) |
| Database Import | LDIF | LDIF | ✅ (Data manipulation automated in openldap2ds) |
| Password Hashes | Varies | Varies | 🔨 (Some types may work, some may need development or migration) |
| oldap 2 ds repl | - | - | ❌ (No mechanism for openldap to replicate to 389 is possible) |
| ds 2 oldap repl | - | SyncREPL | 🔨 (Mostly complete, tests in place, some more QA required.) |
| TOTP | TOTP Overlap | - | ❌ |
| EntryUUID | Part of OpenLDAP | Plugin | 🔨 (Accepted, pathway to rust-by-default underway) |
From this snapshot we can hopefully see that it’s not possible to mix and match openldap with 389 ds, and that there will always be migration steps required. The only feasible mix and match will be if we develop support for 389-ds syncrepl to impersonate openldap’s syncrepl provider (we will likely not consider delta syncrepl). Even in this scenario, as some plugins and configurations will have different attribute values,
This is a set of example topologies we consider we could support migration from
In this scenario, we are able to do a “once off” migration, replacing the OpenLDAP with 389-ds
In this scenario, as we have no method of replication sync, we would consider the migration to be where a parallel 389-ds infrastructure is built, and then the related schema and database is migrated into the new 389 topology.
In this scenario, this is the same as the multiple active RW server scenario. If there are many RO servers, it may not be feasible to buld this fully, so we could have the RO replicas read from 389-ds instead via syncrepl with some live data manipulation. The scope of this is still under investigation.
In this scenario, 389-ds would be built in parallel to use winsync plugin. This would use a different configuration and set of integrations to provide the AD read sync services.
To support the inbuilt openldap schema, 389-ds is should adopt the rfc2307compat schema, which resolves a set of schema conflicts that may arise during a migration. There are no other known conflicts with openldap schema.
A tool called openldap2ds is being developed that can check for and migrate schema to 389-ds from openldap instances. It generates a migration plan which can be reviewed and modified, that then applies changes to the 389-ds instance, as well as suggesting changes for other instances in the topology.
Openldap2ds can also detect some plugins from openldap and configure 389-ds to match. This feature is limited.
Given an ldif export openldap2ds will be able to import an openldap backup into 389ds, performing needed data manipulation to remove openldap specific attributes.
EntryUUID as a plugin for 389-ds - openldap uses this as part of it’s replication model, and many applications are configured to use it as a primary key. If there is an existing parallel 389-ds and openldap instance that will merge, the nsUniqueId and entryUUID will differ, so EntryUUID is developed as a plugin seperate to nsunique id. This adds some duplication in the server, but in return admins do not need to manipulate their entryUUID -> nsuniqueid values, and we don’t need to write another virtual attribute transformer. It also allows admins to post migration change the entryUUID if required.
We do not support saslauthd so any site using this will need to configure pam pass through auth with sssd or other modules.
We do not support TOTP, so this can not be migrated at this time. The versions of openldap on the enterprise distros do not support TOTP anyway, so this is likely not to be an issue.
389-ds does not function as an LDAP proxy, we prefer to have extra read only servers created.
389-ds can use winsync to consume AD information and changes, rather than acting as a proxy.
High Level Workflow
The administrator should inventory their current assets, dataflow and configuration. For example it could be an external IDM that feeds data into silos of OpenLDAP and AD, or it could be OpenLDAP in proxy mode to AD. It could also be pure OpenLDAP.
The replication topology should be accounted for in this step.
It should also be noted which plugins are enabled, and how authentication is performed and where credentials are stored.
The parallel 389-ds instance or topology should be brought up. Test migrations with openldap2ds should be performed. openldap2ds should be run against each server in the topology OR the migration commands should be run manually against each node to ensure they are in the same configuration state.
The openldap2ds tool can be run to import and manipulate the openldap database as needed.
If openldap is acting as a proxy to AD, 389-ds should be configured with winsync. It should be checked that the correct data associations are made to the entries.
It may be required to run both openldap and 389-ds silos in parallel due to the nature of the data and topology. It is critical that writes are able to be directed to BOTH openldap and 389-ds silos OR that they only flow to one or the other and a cutover process is defined.
When the cut over is made in a single data flow scenario, when moving the 389-ds as the incoming write silo, then the remaining openldap stack should be configured to syncrepl from 389-ds OR the entire 389-ds topology should be pivoted to in a single change (this depends on our ability to improve sync repl). This depends on the size and scale of your topology to whether a “single action” cutover is possible.
Rollback plans should always be developed and understood as in any migration. Backups should always be taken as needed (Such as system state backup on windows, ldifs and config on openldap, and db2ldif on 389-ds).
A requested feature has been the ability to have 389-ds be able to provide changes to openldap read-only replicas. It is not feasible to have 389-ds consume from openldap, but to provide is a simpler matter.
Focusing on syncrepl (not delta-sync repl) makes for a simpler, and easier implementation, as well as only focusing on the “refreshOnly” mode. This has largely been implemented already, but some deviations from the current implementation are observed. (See also https://tools.ietf.org/html/rfc4533)
The initial replication is initiated by a search request with the syncRequestOID, SyncRequestValue with a cookie containing the consumers rid and no CSN.
rid=123
The provider then responds with all entries, with a syncStateOID control that has the state add (1) and the EntryUUID of the entry.
The searchResDone message then provides a SyncDoneOID control with a cookie the rid of the consumer, and the CSN that was replicated up to. The cookie format
rid=123,csn=20200525045810.162381Z#000000#000#000000
rid=123,csn=20200525051329.534174Z#000000#000#000000
The OpenLDAP consumer will parse this CSN to order the events that have just been provided, however the consumer then assigns it’s own internal CSN to the events. This CSN is based on the systems localtime and is not a lamport clock, which means that there may be occasions the generated CSN could be out of order relative to others.
The OpenLDAP consumer will assert that the Ldap Message SyncUUID is equivalent to the EntryUUID if EntryUUID is present (if EntryUUID is not present, it is generated from the SyncUUID).
The subsequent requests are made with the cookie with the last rid and csn provided in the SyncRequestValue. This allows the provider to know where to start to “send updates” from. These updates are documented below based on the causing operation type.
We can already provide sync repl with the following setup.
dsconf localhost plugin retro-changelog enable
dsconf localhost plugin retro-changelog set --attribute nsuniqueid:targetUniqueId
dsconf localhost plugin retro-changelog add --attribute entryuuid:targetEntryUUID
dsconf localhost plugin contentsync enable
dsconf localhost plugin contentsync set --attribute syncrepl-allow-openldap:on
dsconf localhost backend create --suffix dc=example,dc=com --be-name userRoot
dsidm localhost initialise
create a sync capable user.
dn: cn=syncrepl,ou=services,dc=example,dc=com
objectClass: top
objectClass: organizationalRole
objectClass: nsAccount
cn: syncrepl
userPassword: password
nsSizeLimit: -1
nsLookThroughLimit: -1
nsTimeLimit: -1
nsidletimeout: -1
#
ldapadd -f 389_sync_acct.ldif -D 'cn=Directory Manager' -w password -x -H ldap://127.0.0.1:3389
We want to limit what can be synced. We achieve this with access controls on the account.
dn: dc=example,dc=com
changetype: modify
add: aci
aci: (targetattr != "aci || creatorsName || modifiersName || createTimestamp || modifyTimestamp || parentId || entryId || nsAccountLock || numSubordinates || tombstoneNumSubordinates || nsSizeLimit || nsLookThroughLimit || nsTimeLimit || nsidletimeout")(targetfilter="(objectClass=*)")(version 3.0; acl "Enable sync repl full read"; allow (read, search)(userdn="ldap:///cn=syncrepl,ou=services,dc=example,dc=com");)
#
ldapmodify -f 389_sync_aci.ldif -D 'cn=Directory Manager' -w password -x -H ldap://127.0.0.1:3389
Add the dsee.ldif
include /etc/openldap/schema/dsee.schema
Load MDB
moduleload back_mdb.la
Configure the database
database mdb
suffix "dc=example,dc=com"
rootdn "cn=Manager,dc=example,dc=com"
rootpw password
directory /var/lib/ldap
index objectClass eq
index entryUUID eq
syncrepl rid=123
provider=ldap://172.17.0.2:3389
binddn="cn=syncrepl,ou=services,dc=example,dc=com"
bindmethod=simple
credentials=password
searchbase="dc=example,dc=com"
# It is likely wise to limit this to objectClasses you are interested in (IE ou, group, account only)
filter="(objectClass=*)"
schemachecking=off
scope=sub
type=refreshOnly
retry="3 +" interval=00:00:00:03
updateref ldap://172.17.0.2:3389
check the changelog:
ldapsearch -H ldap://172.17.0.2:3389 -b cn=changelog -D 'cn=Directory Manager' -x -w password
Show the current openldap cookie:
ldapsearch -H ldap://127.0.0.1 -b 'dc=example,dc=com' -s base -x contextCSN
# example.com
dn: dc=example,dc=com
contextCSN: 21000101110148.000000Z#000000#000#000000
Due to OpenLDAP’s replication being “whole entry” based, and the details in the syncrepl being very simple, it is easy to emulate with a small number of changes. This allows a “readonly” OpenLDAP server to exist, however, access control and bind may not work as “expected”. It may be the case that admins do not wish for this functionality to exist, so OpenLDAP sync is enabled per server.
If we detect the initial request with a rid= format, and the value syncrepl-allow-openldap=on is set, we know that we are in openldap provider mode (compared to other sync repl clients that will provide an empty sync cookie).
We can generate an OpenLDAP compatible CSN by converting the changenumber into a timestamp via unix epoch. By adding an offset we can guarantee that the CSN’s we generate will “always be higher” than anything OpenLDAP can generate, preventing conflicts.
This would allow us to construct an openLDAP compatible synccookie, that has the correct timestamps and that when they come to query us next, we are able to provide “what changed since” and guarantee correct ordering of events.
No other changes are needed.
To enable OpenLDAP sync, the following parameter exists:
cn=Content Synchronization,cn=plugins,cn=config
syncrepl-allow-openldap: off|on
With dsconf:
dsconf localhost plugin contentsync enable
dsconf localhost plugin contentsync set --attribute syncrepl-allow-openldap:on
The currently syncrepl plugin uses nsUniqueId to provide the SyncUUID for operations. As OpenLDAP enforces that SyncUUID must be equal to EntryUUID, when providing to an OpenLDAP client, then we enforce that EntryUUID must be present. This is ensure the UUID’s are stable between OpenLDAP and 389-ds for consuming applications and to prevent data inconsistency between 389-ds and OpenLDAP. Consider if we had an entry with NO entryUUID. When sent to OpenLDAP, the SyncUUID would be from the NsUniqueId, which would cause OpenLDAP to create the EntryUUID as the derivative of the SyncUUID (nsUniqueId). If the entry then had the EntryUUID attribute added, we would not send a delete of the nsUniqueId, and we would send a present of the EntryUUID as the SyncUUID. This would cause the OpenLDAP server to reject the SyncRepl due to conflicting DN and mismatching UUID’s. As can be imagined, this is “not fun”. Rather than attempt to track these changes to make it “consistent” it is much simpler to enforce that EntryUUID is required for OpenLDAP syncrepl.
When in OpenLDAP mode, only entries with “EntryUUID=*” are sent to the requesting client. This is added internally to the filter.
To support consistent deletes, we must then track the EntryUUID in the retro changelog so that we can correctly send entry deletes. This is configured in the retro changelog with:
dsconf localhost plugin retro-changelog enable
dsconf localhost plugin retro-changelog set --attribute nsuniqueid:targetUniqueId
dsconf localhost plugin retro-changelog add --attribute entryuuid:targetEntryUUID
Since we only send entries that have “EntryUUID=*”, we can assert that targetEntryUUID must also then be present in the Retro Changelog. This allows on delete events, for the targetEntryUUID to be sent and the SyncUUID to be consistent.
OpenLDAP 2.5 has a DSEE compatible sync mode that combines syncrepl searches with a cn=changelog search and interpretation. However, the major enterprise distros are using version 2.4 so this is unlikely to be an option.
William Brown wbrown at suse.de