Hi there AppWorks fans,

Welcome to a new installment of AppWorks tips.

Today is the day we finally grab the time to make a deep dive on the connection between our beloved AppWorks platform, and the Documentum platform. For this we will use the ‘Document Store Connector’ which is delivered with the AppWorks platform. I must admit, I played already a couple of times with the connector, but it never ‘clicked’ together all by myself (for some strange reasons!?). From an insider knowledge tip I was told to really make it another try as the connection is made much simpler. Where I learned to use 2 OTDS tenants with ‘Auto-Provisioning of Accounts’ together with OAuth handlers it should now be much easier with just 1 OTDS tenant and an OAuth client ID/secret setting…Let’s dive into this mystery of information and see how far we can get…

Let get right into it…

Before we continue we’ll make a list of things we need to have up and running before we can start this post. It might be a pretty technical list, so if it doesn’t ring a bell I would recommend some research (it’s all explained on this blog site…have a search on the topics):

• AppWorks platform (TomEE) connected via OTDS (‘appworks’ partition)
  • URL (via OTDS): http://192.168.56.107:8080/home/appworks_tips/?authID=OTDS
  • Account: awdev@appworks (with the ‘Developer’ role applied!)
• Documentum (Connection broker, repository and Java Method Server)
  • URL JMS: http://192.168.56.107:9060/DmMethods/servlet/DoMethod (default 9080, but we moved it!)
  • You can also check this URL: http://192.168.56.107:9060/dmotdsrest
  • As well as this URL: http://192.168.56.107:9060/OTDSAuthentication/servlet/authenticate
• Client for Documentum (can be Documentum Administrator on Tomcat, bus this post will also deploy Documentum ReST as valid client)
  • URL: http://localhost:8080/da (I run it outside my VM as we don’t need it continually!)
  • Account: sysadmin
• OTDS (1 tenant where OpenDJ needs to start manually…Just for me, so I don’t forget!) 😎
  • URL: http://192.168.56.107:8181/otds-admin/
  • Account: otadmin@otds.admin

Documentum ReST deployment

The document store connector we will bring to life on our AppWorks platform has a dependency with the exposed ReST layer of the Documentum platform. By default, this is not installed, so we first need to download it. I have Documentum 21.2 up and running and will also use the same version for the ReST client (dctm-rest.war).

Next step is to update some files in this WAR file. I open the file with 7-Zip so I’m able to update these 2 files:

\dctm-rest.war\WEB-INF\classes\dfc.properties

This makes sure our ReST client ‘knows’ where our Documentum repository is running.

1
2
3
4
5
6
7
8
9
10
11

dfc.docbroker.host[0]=192.168.56.107
dfc.docbroker.port[0]=1489
dfc.globalregistry.repository=aw_repo
dfc.globalregistry.password=dm_bof_registry
dfc.globalregistry.username=dm_bof_registry
dfc.session.secure_connect_default=native
dfc.date_format=dd/mm/yyyy hh:mi:ss
dfc.tracing.enable=false
dfc.exception.include_id=false
dfc.exception.include_decoration=false
dfc.session.secure_connect_default=try_native_first

\dctm-rest.war\WEB-INF\classes\rest-api-runtime.properties

This makes sure our ReST client ‘knows’ how to authenticate via OTDS.

FYI: The document store connector makes sure we use the same OTDS authenticated user account for the AppWorks platform as well as the Documentum platform! That’s technically a better solution for security than the use of a so-called ‘service-account’!

1
2
3
4

#See the Documentum ReST Services Development guide!
rest.security.auth.mode=ct-otds_token-basic
rest.security.realm.name=com.documentum.rest
rest.security.otds.login.url=http://192.168.56.107:8181/otdsws/login

Our WAR file is prepared, time to deploy it on our JMS. This is just an upload action to the VM in this location: /opt/documentum/tomcat9.0.43/webapps/

Tomcat will ‘explode’ the WAR file to a ‘webapps’ directory. After a minute you should be able to access these URLs:

• http://192.168.56.107:9060/dctm-rest

• http://192.168.56.107:9060/dctm-rest/repositories

  Pro-TIP: Add .json in the URL: http://192.168.56.107:9060/dctm-rest/repositories.json and make sure you have a ‘JSON formatter’ plugin installed in your Chrome browser. This makes it much easier to ‘browse’ through the JSON “href” URLs!

• http://192.168.56.107:9060/dctm-rest/repositories/aw_repo.json (will give an ‘Authentication failed’ for now as we didn’t connect OTDS (yet!))

  If you already want to browse around, you need to comment those 3 lines in the ‘rest-api-runtime.properties’ and do a restart of the JMS systemclt restart dm_jms. After restart, you will be able to authenticate via the non-OTDS account ‘sysadmin’.

Hookup Documentum with OTDS

In this step we’ll make sure our Documentum platform is able to authenticate via OTDS. For this we use IAPI with a command like this: /opt/documentum/product/21.2/bin/iapi aw_repo

Login with the installation owner account ‘sysadmin’, and execute these script lines

1
2
3
4
5
6
7

retrieve,c,dm_server_config
append,c,l,app_server_name
OTDSAuthentication
append,c,l,app_server_uri
http://localhost:9060/OTDSAuthentication/servlet/authenticate
save,c,l
reinit,c,aw_repo

c = current session

l = last retrieved object id

quit = to return to bash

For you to understand what should happen:

Next…

Configure OTDS

Now, we do exactly what we “normally” would do for our AppWorks platform to authenticate with OTDS. We create a new resource (this time a resource for Documentum), and we add the already excising ‘appworks’ partition as access role. So, in the end we have 1 partition with users/groups which is used in 2 resources (one for Appworks itself and one for Documentum)

Let’s dive into OTDS http://192.168.56.107:8181/otds-admin/ and login with ‘otadmin@otds.admin’. Go to the ‘Resources’ category and ‘Add’ a new resource:

Use this information:

• General tab

  • Resource Name: dctm_resource

• Synchronization tab:

  • Mark “User and group synchronization”
  • Synchronization connector: REST (Generic)

• Connection information:

  • http://192.168.56.107:9060/dmotdsrest

  • Username: aw_repo\sysadmin

• User Attribute Mappings tab:

  • Set the client_capability = 2 for the users (so, they will be ‘contributor’ in the repository of Documentum)

Now do a save of the resource and copy the ID: dd83d441-6a52-42e6-b7bd-177ce0bea4e0

!!DON’T HIT THE VERIFY ACTIVATION BUTTON!!

Normally we activate a resource via the AppWorks platform when we configure OTDS in the ‘Security Administration’ artifact…remember?

Only from a Documentum perspective we don’t have this option (yet!?) via a nice UI. For Documentum, we need to make a manual HTTP POST to this URL (where the ID is our just copied resource ID!):

http://192.168.56.107:8181/otdsws/rest/resources/dd83d441-6a52-42e6-b7bd-177ce0bea4e0/activate

We can use a tool like Postman to make this call, but I downloaded CURL for Windows so I’m able to execute a prompt command like this:

curl -X POST http://192.168.56.107:8181/otdsws/rest/resources/dd83d441-6a52-42e6-b7bd-177ce0bea4e0/activate

You will get a response in JSON format. Save that retrieved ‘secretKey’:

1

{"secretKey":"mUdiaFjooL9i4AqfpeVQwQ=="}

Optional: In OTDS you can now also ‘Verify’ the activation status for this activated resource!

Add partition into the access role

With the creation of the resource there is also a corresponding ‘Access Role’ available in the ‘Access Roles’ category. We set the option (optionally) to ‘Include groups’ and after this hit the ‘View Access Role Details’:

In the ‘User Partitions’ tab we will ‘Add’ the already available ‘appworks’ partition:

Don’t forget to hit that ‘Save’ button!!

After this we can check the logging tail -999f /opt/documentum/tomcat9.0.43/logs/dmotdsrest.log for User 'awdev' has been created in the docbase and check the Documentum repository with a ReST call:

http://192.168.56.107:9060/dctm-rest/repositories/aw_repo/users/awdev.json

If you can’t access it you can also check with the good old IDQL tool: /opt/documentum/product/21.2/bin/idql aw_repo

Login again with the ‘sysadmin’ account and run a DQL statement like this:

select user_name, user_login_name, client_capability, user_source from dm_user where user_name = 'awdev';

Don’t forget to give a ‘go’ after the statement. Give a ‘quite’ to return to bash.

Yes, I know…We could also check with the DA client, but the resources are limited, and now you see a bit more Documentum pro-code! 🤔

Create an OTDS OAuth client (ID/Secret)

If you made it this far you can almost join the “#bind-it-all-together-club”…we’re halfway there…time to create our first client…Why? Well, later on we’ll see the document store connector for Documentum uses an OAuth connection for the authentication part of the configuration. We just pre-sort on what is to come in a later section.

Back to the OTDS admin console again where add a new OAuth client

Use this information:

• General tab
  • Client ID: dctm_oauth_client
  • Confidential = TRUE
  • Sign-out URL: http://192.168.56.107:8181/otdsws/logout
• User partition tab
  • Global is fine, but we can also use our available ‘appworks’ partition
• Advanced tab can be left as is
• Redirect URLs: http://192.168.56.107:8181/otdsws/login

Save the OAuth client and because we made it ‘Confidential’ we receive a popup with the generated ‘client secret’. SAVE IT (as you will only see this once!)

Client secret I received: QZxvH2l81Fp6rxE7ho23vY8xVYTdz15Q

Enable OTDS on JMS

We’re getting closer and closer…

Let’s edit an interesting file on the server: vi /opt/documentum/tomcat9.0.43/webapps/OTDSAuthentication/WEB-INF/classes/otdsauth.properties

The first property ‘certificate’ is filled with a value retrieved with this URL: http://192.168.56.107:8181/otdsws/rest/systemconfig/certificate_content

This is my final file content:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

#Make a call to OTDS as <OTDSWS_URL>/rest/systemconfig/certificate_content (eg. https://otds_app_host:8443/otdsws/rest/systemconfig/certificate_content). Copy the content between BEGIN CERTIFICATE and END CERTIFICATE and paste it as a single line below
certificate=MIIC3DCCAcSgAwIBAgIJALAITzm0RySkMA0GCSqGSIb3DQEBCwUAMA8xDTALBgNVBAMMBE9URFMwHhcNMjEwNDI1MjA0NDMyWhcNMjIwNDI1MjA0NDMyWjAPMQ0wCwYDVQQDDARPVERTMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEApoZ2dNGjOfVdNJu10czoP8ohO6yhTqVIktsmZJ0GNbDhaQUmiBKiPJGJhdqXhl+olkJryJENZbBudQyv5YBZHLhhdRnma5N1imav9GJKshSn6Xb0M8SOqIeUi3cVHKV+WLZ7rZ6TpNh4p/lfuD4apyTSIL4XW5nmUQp3aW4eTrcJeMdK5GOdF8O84PFdgteSWMkRG2pqC9K9ZadzcpN/Vx92KqHOtsXPRZNh8qmAfPwUGWCjtZFaTFyB2XqbVNKUS+dMy2K72juTzbCYxpaqLlppudRexxG+cUXBjAIU7IfmCiyIlTTyJKR+8qIiSPZTmTmK3Wnx26Fd1BHAHnsGyQIDAQABozswOTAPBgNVHRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIFoDAWBgNVHSUBAf8EDDAKBggrBgEFBQcDATANBgkqhkiG9w0BAQsFAAOCAQEADaGUCo90JAATxzRVMyyS5HRRskRwKv9yUUEoVN/EZJIS3bfrvLBxRysKXroAm39OVjVLa5paLQh1rvUaqCKx6VavT4hQvE3St9BqDcUVGa3ODfAbCglIn/bKcJ/lDsNcGNjXAQymkT969PuSbJZyST/02tYkF1paZm4aciVHkkmRRXC7WttkcnpH10NxrEIZ3MTIO+KYkYorjqzEVe2RE7gRl3H+4iV9W7HBe3ge878TpSq2ZJQkfDyWBChOvOeiKJyeTXRtmeBbal6AB3eZhWyRQb9Rc4pH6QIM1WlJMiPwjDio3pe4Wusr9bKq04ohAxLUEGFKhmMUvvi106OcUA==

#Configure otds_app_host and otds_app_port below. The below REST call is used to authenticate username and password credential with OTDS
otds_rest_credential_url=http://192.168.56.107:8181/otdsws/rest/authentication/credentials

#make this value to true if the authentication happens through OTDS but users not synced from otds
is_hybrid=false

#Configure otds_app_host and otds_app_port below. The below REST call is used to authenticate OTDS ticket with the resource Id and secretKey configured for the docbase
otds_rest_ticket_url=http://192.168.56.107:8181/otdsws/rest/authentication/resource/validation

#Configure otds_app_host and otds_app_port below. The below REST call is used to authenticate using Oauth2 Token obtained by passing username and password credential with OTDS
#Set this flag to true if the authentication needs to be done through Oauth2 and configure client_id and client_secret as well
passauth_use_oauth2_token=false
client_id=dctm_oauth_client
client_secret=QZxvH2l81Fp6rxE7ho23vY8xVYTdz15Q
otds_rest_oauth2_url=http://192.168.56.107:8181/otdsws/oauth2/token
synced_user_login_name=sysadmin

#configure docbase name and its respective  resource Id and secret Key below
aw_repo_resource_id=7489c8a5-faa8-4a48-98cb-c4f6a4fc8147
aw_repo_secretKey=mUdiaFjooL9i4AqfpeVQwQ==

#The below parameters are used to automatically refresh the certificates when certificate roll over happens
auto_cert_refresh=false
cert_jwks_url=http://192.168.56.107:8181/otdsws/oauth2/jwks

After change, we need to restart the JMS via systemctl restart dm_jms

In the meantime we can track some logging if you like:

tail -999f /opt/documentum/tomcat9.0.43/logs/dmotdsrest.log

tail -999f /opt/documentum/tomcat9.0.43/logs/otdsauth.log

tail -999f /opt/documentum/tomcat9.0.43/logs/catalina.{date}.log

It’s getting more and more interesting…next step!

Test Documentum authentication via OTDS

Finally!…Time to test our first authentication!

Connect to IAPI: /opt/documentum/product/21.2/bin/iapi aw_repo with sysadmin superuser account

Now give this API statement connect,aw_repo,awdev@appworks,dm_otds_password=admin and we should be able to start a new session for this user with the Documentum repository…GREAT!!

I tried this call before we configured JMS with OTDS and this call is NOT working until JMS is prepared (I thought it would work already as the awdev.json already gave result from the ReST call earlier for some strange reason, but that’s probably updates from my side!?)

Now for the OAuth authentication: connect,aw_repo,awdev@appworks,dm_otds_oauth=<access_token>

The access token can be retrieved via this URL call in the browser: http://192.168.56.107:8181/otdsws/oauth2/auth?response_type=token&client_id=dctm_oauth_client&client_secret=QZxvH2l81Fp6rxE7ho23vY8xVYTdz15Q&redirect_uri=http://192.168.56.107:8181/otdsws/login

Make sure to pass the correct parameter values for the ‘client_id’ and ‘client_secret’, but you have that information available now!

The URL will redirect you to the OTDS authentication screen where we need to login with our ‘awdev@appworks’ credentials. When done we will retrieve an ‘access_token’ as parameter in the URL which can be used in the API connect call…With a satisfying API session result: ‘s1’

Hooray…Call yourself PRO when you made it this far! 🤓

Test dctm-rest OAuth via OTDS

Now, let’s see if we can also use that same OAuth token to connect to our ReST layer of Documentum as that is eventually are ultimate goal to make the document store connector to do its job.

So, open up your browser and go to this URL: http://192.168.56.107:9060/dctm-rest/repositories/aw_repo/types.json?recursive=true&parent-type=dm_document&access_token=

You see in the end the ‘access_token’ as parameter! Well, where are you waiting for!? Copy that retrieved token from the previous section behind it and see if it works!

Ohwwww mama….Hallelujah…Shining bright like a diamond! Beer time… 🍺

Document Store Connector (to Documentum)

Well…This last one is now just a piece of cake as all the rest is already prepared for a smooth connection!

Open up AppWorks in design time in your own organization, start up the ‘System Resource Manager’ artifact, and create a new connector of type ‘Document Store’

Hit ‘Next >’

Give an interesting name and select all the web service interfaces:

‘Next >’

Make it start up automatically:

Hit ‘Next >’ again…

Fill-in all the data you have available by now:

We leave that ‘Viewer Type’ for what it is by now…We already posted about Brava!, and the new Intelligent viewer is on the backlog.

Hit ‘Next >’ and ‘Finish’ it in the final screen…

The service container will be created and should start up with a green lighted icon…Like the other containers.

If not, you might have a look in the logfile: sudo tail -99f /opt/opentext/AppWorksPlatform/defaultInst/Logs/appworks_tips#documentstore#document_store.xml

After this change we make a call on two URLs:

• http://192.168.56.107:8080/home/appworks_tips/app/documentservices/rest/cache/flushOrgEntry

• http://192.168.56.107:8080/home/appworks_tips/app/documentservices/rest/cache/reset

Why? Well because we already had some experience with the platform and other document store connector configurations…So, it’s just precaution and for you to know it’s there!

Entity with Content BB

We’re getting close people…Let’s continue the grind as now we need to be able to upload some content to the platform through the AppWorks runtime where the brand-new store connector will upload the file to Documentum.

So, back into design time, where we start with a simple project structure where we will create a new ‘entity’ in our ‘entities’ folder.

We will get a modal popup for the entity to add new properties where we just create 1 property ‘doc_name’ with label ‘Name’ and type ‘Text’…Nothing fancy as long as you mark the option to generate all the default interface BBs.

Click the ‘Add’ button and save the entity with name ‘document’. You can make it as fancy as you like, but for now it’s important to add the ‘Content’ BB and update the ‘DefaultLayout’ with some content related panels:

Save the entity again and do a publication and quickly check if you can create a new ‘document’ entity instance (without any content!!)

Don’t forget to make sure you test with the account which is authenticated against OTDS. I use this URL:
http://192.168.56.107:8080/home/appworks_tips/app/start?authID=OTDS

My user account is ‘awdev@appworks’

AppWorks admin content configuration

There is a reason why I mentioned to not (yet!) upload any content. That’s because we need to tell our ‘Content’ BB (on the just created entity) what it needs to create on the Documentum platform when we upload content in Runtime…Read that line again if the penny hasn’t dropped! 🤔

Let’s jump into the ‘/app/admin’ of our platform (with the OTDS authenticated user!):
http://192.168.56.107:8080/home/appworks_tips/app/admin?authID=OTDS

Click on the published project and open the ‘Content configuration’ for the ‘Document’ entity…

Hmmm…That’s a circling around!

Let’s check some logging: sudo tail -999f /opt/tomcat/apache-tomcat-8.5.43/logs/directory-access.log

That’s our first error: "Impersonation failed: com.opentext.otds.OtdsException: Impersonation not enabled on resource 7489c8a5-faa8-4a48-98cb-c4f6a4fc8147."

Back to OTDS administration and enable this option on both our resources.

From experience, I know there is also such ‘impersonation’ option on the OAuth client!

Impersonation!…What magic does it do? #RTFM…This time the OTDS administration guide. We also see that impersonation also needs to be enabled when asked by the application…Well, our logfile was pretty clear!

Now, let’s refresh that last screen of the ‘/app/admin’ and there you go…We clearly see Documentum stuff passing by!

This screen lets us configure where type of document (also a subtype of dm_document with custom properties is possible although you need to make it available in Documentum first, but that’s another story!)

First it’s time for a small party… 🍰 🍹 🎉 🎈

Next!

A-Z test with validation

Back to the AppWorks runtime with our OTDS user, open or create an instance of the ‘Document’ entity and upload some content!

Nice…It gets uploaded, let’s check what happened in Documentum with IDQL!?

/opt/documentum/product/21.2/bin/idql aw_repo…login with the ‘sysadmin’ account.

Let’s start with this query (don’t forget to give a ‘go’):

select r_object_id, i_folder_id, r_lock_owner, object_name, title, subject from dm_document where title = 'hello title';

Well…1 result (r_object_id = 0912d59180002122) as expected!

My DQL is a bit rusty, but my experience helps me along with a statement like this (You never forget the good things in life!)

1
2
3
4
5
6

SELECT DISTINCT d.r_object_id, d.r_lock_owner, d.object_name, d.title, d.subject, f.r_folder_path
FROM dm_document d, dm_folder f
WHERE any d.i_folder_id = f.r_object_id
AND d.title = 'hello title'
AND f.r_folder_path IS NOT nullstring 
enable(row_based);

Now we also see the folder path passing by where our document is saved in this location: /Temp/appworks_tips/appworks_tipsgeneric/document/1/Contents/Default

Is this a logic location?

Well, the ‘/Temp’ is for sure as we configured it in the document store connector…The rest? Have a second look on the ‘Content’ BB for the entity in design time:

${system.organization}/${solution.name}/${contentbuildingblock.path} brings us closer to the path we’ve just seen…Nice!

What else?

Let’s have a look with ReST to see if we can get a properties overview of this document in JSON format. Check this URL (where the r_object_id is our created document in Documentum):
http://192.168.56.107:9060/dctm-rest/repositories/aw_repo/documents/0912d59180002122.json

Some interesting bullets we see passing by:

• The “Title” seems to have a mapping with the ‘object_name’ in Documentum
• The metadata ‘Name’ information is lost in space, but I expected to be the ‘object_name’ again
• The metadata ‘Title’ and ‘Subject’ are a correct match
• The ‘owner_name’ of the document is our ‘awdev’ account…As expected and what we also want!

If you get an authentication failure!? Grab a new access_token with this URL http://192.168.56.107:8181/otdsws/oauth2/auth?response_type=token&client_id=dctm_oauth_client&client_secret=QZxvH2l81Fp6rxE7ho23vY8xVYTdz15Q&redirect_uri=http://192.168.56.107:8181/otdsws/login and pass it as parameter in the ReST URL (like we did before in this post)

What else?

Trigger some action in runtime…Like a checkout!?

After a check of the previous DQL statement we see indeed the ‘r_lock_owner’ being filled with ‘awdev’…GREAT! I leave the other actions for exploration by you, but for sure check the ‘Properties’ action…

…as well as the action ‘Link document’ behind the ‘plus’ sign…

Yes…you see it correctly! That’s a small Documentum repository browser…In AppWorks!

We’ve just passed the golden achievement! 🥇🥇🥇 and we finally nailed it…It’s a well-earned ‘DONE’ for all the people who would like to connect the good old Documentum platform to the good new AppWorks platform. It’s the best of 2 worlds in my opinion, and we’ll keep on grinding to keep it that way. What a journey to explore where we also brought our Authentication knowledge to a next level. Do you have any question about this pretty hard blog with lots of technical dept? Just let me know in the comments. Have a great week-end, and I see you in the next one.

Don’t forget to subscribe to get updates on the activities happening on this site.