Hi there AppWorks fans,

Welcome to a new installment of AppWorks tips.

This post will demonstrate several ways of exposing entity data to the outside world. This can be in the end-user runtime, in the Typed ReST section of the admin portal ‘/app/admin’, or just via the ‘/app/webservices’ portal of the platform. Which one to use? It all depends, but all have their advantages and disadvantages. I split the post into 2 sections; A part for SOAP and a part for ReST…Let’s have a look!

Let get right into it…

We start with a fresh start image (via a snapshot restore on my machine!); Boot it and login to your workspace and project. For this post we only need an entity and as many building blocks as possible to expose data from it. I’m curious about what is exposed and what is not exposed in specific circumstances! So, I low-code myself a ‘Project’ entity with different type of properties (not all types are required; It’s just that we get some random data):

All these properties are visible in the ‘DefaultList’ which I renamed to lst_active_projects. I remove the description and order the columns a bit, so the logo gets upfront; you can move the rest as you think it looks nice for your end-users! 😄 After the reorder, I make sure to filter out the non-active projects from the list:

Next step to add interesting BBs (for this post):

• ‘Relationship’ BB: Keep it simple with just a ‘hasChild’ relation to a new ‘Contact’ entity where you add some interesting contact information like ‘name’ and ‘address’.
• ‘Assignee’ BB: Assign it during creation to the ‘User.UserId’ property
• ‘Lifecycle’ BB: Create a simple one with just one state ‘Finish’; No further activities required
• ‘Tracking’ BB: Exposes creation/modify data
• ‘File’ BB: For just one file upload
• ‘History’ BB: To get hands on audit data if needed; I low-code the logfile with this name: hl_project (saved nicely in a configs folder of the current project!)

Finally, I created one extra ‘List’ BB; It’s an export ‘List’ BB, exposing all entity data possible. Name the list lst_export_projects and mark all possible properties available (incl. the building block related properties where you think it’s interesting!). My list is in total exposing 35 columns; I know, it’s an unreadable list in the UI, but that’s not the point of this exercise. Well, it’s a solution as long as you keep it separate in some “export” part of the normal implementation…Don’t mix it with the current solution unless explicitly requested by your customer and it makes sense!!

In my AppWorks solutions I always see these types of lists extensively used; You can use prefixes in the name to separate them in design time

• Internal lists for designers behind grids or repeating groups; prefix int_
• Runtime lists for your end-users; A prefix is not needed as I see it as the “default”!
• Export lists to have a download to XLS/CSV; prefix exp_
• External lists not for runtime, but called from external systems; prefix ext_

After an update on the ‘Create’ form and a publication you should be able to create that first couple of projects:

The images are from https://getavataaars.com

This is my starting list:

You don’t want to see my export list in runtime now; This is also not needed as we just want to compare the outcome of this list with other export possibilities in the next steps!

From a runtime perspective, the end-user can already export our list to Excel or CSV file for further managing the data (if applicable). Only, we want to explore what else is possible and what are the limitations or advantages.

SOAP

For this part, I see two ways of exposing the export list of data to the outside world. The first one is via the ‘Web service’ BB where we expose data over a new operation to get all entity instances. The second is over a BPM with an output message. We can call this BPM via the ‘executeProcess’ webservice, or we can generate a dedicated webservice out of it (out of scope for this post)!

Webservice BB and call via services portal

We start by adding the ‘Web service’ BB to our ‘Project’ entity, and we also add a new search operation get_all:

There is no need to configure it further; A “blackbox” set of data is exposed which we are going to find out! Now do a first publication for the entity, but before we dive into runtime, we need to make sure to have a service container up and running of type ‘Application Server Connector’. I leave this task with you; It’s explained several times already in other posts, but comment me when you don’t have a clue!?

With the service container up and running from the ‘System Resource Manager’ artifact, we can now have a test in runtime via the ‘Services’ portal. WHAT? Yes, the services portal is behind this URL: http://192.168.56.107:8080/home/appworks_tips/services

I know we can also do this from a ‘Webservice Interface Explorer’ artifact, but It’s good to know other paths are available too!

We can trigger a call by searching the entity, hit the ‘Test’ button, and ‘Send’ out a request:

Now the great question: What data is exposed, and does it match our export List BB!? Well, my entity instances are indeed exposed (incl. the properties per instance)! What I miss are the specific properties for the exposed building blocks; I only see IDs passing by. So, if you want a specific value from (for example) the ‘Assignee’ BB, you need to make an additional service call to retrieve the rest of the information. Some BBs already expose specific data like the ‘Lifecycle’ and ‘Tracking’ BB.

Advantages:

• It’s an easy way of exposing data
• Out of the box feature without any specific low-code functionality
• Directly callable from the service portal or ‘Webservice Interface Explorer’ artifact

Disadvantages:

• It needs a service container of type ‘Application Server Connector’
• Not all building block information is directly exposed and need a specific second service call
• I miss ‘Identity’ specific details; The ‘ItemId’ and ‘Id’ are exposed, but the rest (like ‘ItemStatus’) is not
• Only technical values, but this can also be an advantage!
• It’s not an export nicely in Excel/CSV format, but plain XML data!

Over a BPM with an output message

In BPM-land we’re totally free of what we can expose as an end-result. We can combine several service calls and output them into 1 response; It’s not my first recommendation to bring such situation in place for your solution, but when calling multiple requests is causing problems (performance wise or technical wise) than this is your way to go.

Before diving into the BPM craftsmanship, we first define the list of webservices we want to call. The first one is the ‘get_all’ service we exposed ourselves; In the previous section we saw what gets exposed via this call. Now we only need to find other related webservices for the missing information on each entity we retrieve. So, have a look again in your lst_export_projects ‘List’ BB to see what related data we would like the call. This is my list of service calls and notes after searching for specific data:

• Title BB: Not needed as it’s already in the response of the ‘get_all’ service.
• Identity BB: Not possible to get all data; I miss for example the ‘IsTemporaryCopy’ and ‘ItemStatus’. It’s also not possible to get it with my exposed ‘ReadProject’ service, and also not from the ‘ReadIdentity’ webservice as this service exposes something else. Interesting, but a limitation for my use-case.
• Assignee BB: For this one I want the additional username and display-username which we can retrieve with service ‘ReadUser’ passing the ‘ItemId’ of the ‘AssigneeIdentity’.
• File BB: Not needed as all its data is already in the response of the ‘get_all’ service.
• Lifecycle BB: It also looks like all data is already in place too!
• Tracking BB: Almost all data is already there except the friendly username for the creator/modifier, but we can grab this information again via ‘ReadUser’.

So, with this overview in mind, it’s a real simple BPM we need to build. Just a service call to ‘get_all’, retrieving the project instances; We loop over it, grab the data for each instance, read the corresponding user via ‘ReadUser’ and expose it via this XML-part to an output message like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

<entity_instance>
    <logo/>
    <name/>
    <prio/>
    <rating/>
    <start_date/>
    <is_active/>
    <duration/>
    <description/>
    <budget/>
    <title/>
    <id/>
    <itemid/>
    <ass_name/>
    <ass_display/>
    <file_name/>
    <file_size/>
    <file_modified/>
    <file_mime_type/>
    <file_storage_ticket/>
    <lc_current_state/>
    <lc_current_state_id/>
    <lc_root_state/>
    <tr_created/>
    <tr_modified/>
    <tr_created_by/>
    <tr_modified_by/>
</entity_instance>

Now for the implementation…From your project, create a new document of type ‘Business Process Model’; I do this directly on my bpms folder and I save the BPM with name bpm_get_all_projects. Add the start- and end-construct with in the middle 2 activities. You can inject the first activity with the service ‘get_all’. On the second activity you can apply a ‘For Each’ loop from the ‘Group By’ context menu. This is your first result (where you also apply a solid name for your iterator…A common forgotten mistake!):

You can do a first run to see the first result, but you should be fine with this first MVP looping over each project instance. Now it’s time to create a new output message, and in parallel also a new element that will include our XML part from above. Open the message map from the bottom tab, and under the ‘Process Specific Messages’ we can create a new message with the name output. You can paste in the XML part as a new element as well. This is the end-result:

Moving back to the model tab, you can now apply the output message to the end-construct:

There are 2 final things left now and that is adding a second activity in the loop (as the first will do a ‘ReadUser’ service call). The second activity will do the mapping from the received data to the entity_instance element and add it into the output message…You’re still with me? 😅 So, like this (with some pro-touch with coloring from your development convention and BPM notes):

Lastly, we need some message mapping in those activities which will result in a consolidated massage map like this:

This is part 2 where the last call adds the complete element to the output message:

Watch the pointer to my iterator name…It is also a common mistake to not use the iterator name!
So, instead of ns2:get_allOutput/ns2:get_allResponse/ns4:project/ns9:Assignee/ns9:AssigneeIdentity/ns10:Identity-id/ns10:ItemId/text() use instance:iter_projects/ns9:Assignee/ns9:AssigneeIdentity/ns10:Identity-id/ns10:ItemId/text()

Save it, publish it, and do that first run with <F12> in the model overview. Check your PIM for the instance and double-check if the output message contains what is configured:

From me a green flag! ✅

We can now call this BPM via the webservice ‘executeProcess’:

1
2
3
4
5
6
7
8
9
10

<SOAP:Envelope xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/">
   <SOAP:Body>
      <ExecuteProcess xmlns="http://schemas.cordys.com/bpm/execution/1.0">
         <type>definition</type>
         <receiver>nl-bos-generic/bpms/bpm_get_all_projects</receiver>
         <message />
         <source>Run from Web Service Interface Explorer</source>
      </ExecuteProcess>
   </SOAP:Body>
</SOAP:Envelope>

We can also generate a new webservice out of it and run it from there (this only requires an additional service container of type BPM what I will not expose in this post now…Comment me on guidance!)

If performance wise the BPM is slow, you can try to change its execution mode to ‘short-lived’. Don’t change all “service” BPMs into ‘short-lived’; If they don’t require any direct feedback in runtime there is no need to do so!

Now the great question: What data is exposed, and does it match our export List BB!? Well, we are totally free here what we expose, but still it’s not possible to expose all data exposed in my List BB lst_export_projects like the 2 specific identity properties! That’s interesting as I always thought BPM has endless possibilities, but with this simple example we reached the border of the platform!? Well, is that really the case!? For sure not, but that requires a totally different approach which you will see when we dive into the ReST section of the platform…And with ReST we bend our implementation to the HTTP connector implementation where we talk JSON instead of XML.

Advantages:

• It’s a free way of exposing data, and you are in control on the data you expose
• You can limit the service calls from the outside application
• It’s still out of the box functionality, but required your BPM low-code skills
• Directly callable by the service portal or ‘Webservice Interface Explorer’ artifact via ‘ExecuteProcess’
• You can generate a new custom service out of it (which depends on the BPM service container)

Disadvantages:

• It needs a service container of type ‘BPM’ (if you generate your own service)
• It still misses some specific properties which I find strange not able to exposing them…Because it’s BPM!…Or maybe I didn’t look deep enough?
• Again, only technical values, but this can also be an advantage!
• It’s not an export nicely in Excel/CSV format, but plain XML data!

OHWWWW…After reviewing my BPM, I found a BUG!! 😣 Do you see it as well? See the consolidated view in the mapping of my data!? Comment on me when you find it too!

ReST

Part two of the post is all about ReST (or is it REST…I don’t get the world!?…We’re talking “Representational State Transfer” here). Right, what about ReST?…Well, we have our entity already in place and ReST is so easy that it’s accessible from just a simple URL in the administration portal ‘/app/admin’:

This is my URL http://192.168.56.107:8080/home/appworks_tips/app/entityservice/aw_tipsaw_generic where the last part aw_tipsaw_generic is a concatenation of my ‘package owner’ and ‘product name’ (see your package properties in the solution in design time)

The ReST portal of the platform is based on the OpenAPI specification and nicely documented with a Swagger UI; Now you know where the familiar colors for the CRUD operations come from, and with this also the possibility to try-out such operation. In the list (which expands with each release of the platform), you can search for lst_export_projects which has a separate GET operation. Try it (without any further input) and check the result…Nicely in JSON format (mine is reduces at the end to show only 1 entry of the in total 4!):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63

{
  "page": {
    "skip": 0,
    "top": 0,
    "count": 4
  },
  "_links": {
    "self": {
      "href": "/aw_tipsaw_generic/entities/project/lists/lst_export_projects"
    },
    "first": {
      "href": "/aw_tipsaw_generic/entities/project/lists/lst_export_projects"
    }
  },
  "_embedded": {
    "lst_export_projects": [
      {
        "_links": {
          "item": {
            "href": "/aw_tipsaw_generic/entities/project/items/16385"
          }
        },
        "Tracking": {
          "CreatedDate": "2023-04-21T14:03:23Z",
          "LastModifiedDate": "2023-04-21T14:03:23Z"
        },
        "Lifecycle": {
          "CurrentState": "Finish",
          "CurrentStateId": "08002700-8c65-a1ed-b813-01fb7cd920d0",
          "ParentState": "Finish",
          "PreviousState": null,
          "PreviousStateId": null,
          "RootState": "Finish"
        },
        "Title": {
          "Title": "project-16385"
        },
        "Properties": {
          "prj_logo": "/aw_tipsaw_generic/entities/project/items/16385/image?name=prj_logo",
          "prj_name": "test001",
          "prj_prio": "medior",
          "prj_rating": 5,
          "prj_start_date": "2023-04-21Z",
          "prj_is_active": true,
          "prj_duration": "P1M",
          "prj_description": null,
          "prj_budget": "10000.0000"
        },
        "Identity": {
          "Id": "16385"
        },
        "File": {
          "FileName": null,
          "FileSize": null,
          "LastModified": null,
          "MimeType": "application/unknown",
          "StorageTicket": null
        }
      },
      ...
    ]
  }
}

Again, the great question: What data is exposed, and does it match our export List BB!? I directly see some missing properties! That’s the ‘ItemId’, ‘IsTemporaryCopy’, and ‘ItemStatus’ for the identity part; For the tracking information, I definitely miss the creator and modifier name! Interesting and not expected; For the identity part I can agree, but the missing tracking information is a #FEATURE_REQUEST (or not?…keep reading…)

Advantages:

• It’s available out of the box
• It produces JSON…The outside world will already love you for it!
• Directly callable from the ReST portal of the platform
• No further service containers are required

Disadvantages:

• We miss some specific properties, but keep on reading…
• Here also, only technical values, but this can also be an advantage!

…

We’re not yet there as I was just wondering what is called in the background of our list when we open it from a runtime perspective. So, have your Chrome Development Console <F12> ready and open the list again from the start of this post! I mean the one I didn’t make a screenshot from because it was a mess with too much data exposed. You will conclude a POST request is done to URL http://192.168.56.107:8080/home/appworks_tips/app/entityRestService/Elements({list_bb_id})/ResultItems where the {list_bb_id} is a fixed ID across DTAP-environments. The request needs a payload in JSON like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

{
   "viewId":"080027008c65a1edb812aec1165f2f18",
   "versionId":"6f08c0bbdc5f3d11b1ef0296caa12a7c",
   "dataSourceId":"080027008c65a1edb811a3225a69ab63",
   "dataSourceIds":[
      "080027008c65a1edb811a3225a69ab63"
   ],
   "parameters":{
      
   },
   "distinct":false,
   "select":[
      
   ],
   "skip":0,
   "top":200
}

The result? Well…It’s an amazing discovery! My missing properties are exposed! It even contains dictionary data with updatability information, type of data, and even Base64 strings of images. This is my output which is cleaned on the dictionary data:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64

{
    "result": {
        "skip": 0,
        "top": 200,
        "count": 4,
        "items": [
            {
                "Tracking": {
                    "CreatedDate": "2023-04-21T14:03:23Z",
                    "LastModifiedDate": "2023-04-21T14:03:23Z"
                },
                "Lifecycle": {
                    "CurrentState": "Finish",
                    "CurrentStateId": "08002700-8c65-a1ed-b813-01fb7cd920d0",
                    "ParentState": "Finish",
                    "PreviousState": null,
                    "PreviousStateId": null,
                    "RootState": "Finish"
                },
                "Title": {
                    "Title": "project-16385"
                },
                "Properties": {
                    "prj_logo": "/home/appworks_tips/app/entityRestService/Items(080027008c65a1edb811a3225a69ab63.16385)/Image?iv=1682085800192&name=prj_logo",
                    "prj_name": "test001",
                    "prj_prio": "medior",
                    "prj_rating": 5,
                    "prj_start_date": "2023-04-21Z",
                    "prj_is_active": true,
                    "prj_duration": "P1M",
                    "prj_description": null,
                    "prj_budget": "10000.0000"
                },
                "Identity": {
                    "EntityType": "080027008c65a1edb811a3225a69ab63",
                    "Id": "16385",
                    "IsTemporaryCopy": false,
                    "ItemId": "080027008c65a1edb811a3225a69ab63.16385",
                    "ItemStatus": 1
                },
                "File": {
                    "FileName": null,
                    "FileSize": null,
                    "LastModified": null,
                    "MimeType": "application/unknown",
                    "StorageTicket": null
                },
                "CreatedBy$Properties": {
                    "Name": "awdev@awp"
                },
                "LastModifiedBy$Properties": {
                    "Name": "awdev@awp"
                },
                "AssigneeIdentity$Properties": {
                    "Name": "awdev@awp",
                    "IdentityDisplayName": "AppWorks Developer"
                },
                ...
            },
            ...
        ]
    },
    ...
}

Well, great stuff, but maybe not that smart to share this POST request to the outside world without explicit instruction on how to use it! That is also where it gets interesting as maybe you’ve noticed already; the ReST portal not only exposed a GET operation for lst_export_projects, but also a POST operation…I have an “AHA”-moment here!!!

So, back in the ReST portal I quickly tried out the POST method for that same list operation where I also see this quote passing by in the description: Handles the queries with large, complex sets of parameters that can be too large to be passed as query parameters. This already matches my assumption; be cautious on large data sets and filter out what’s not needed. Let’s do a first call with this JSON body in the request:

1
2
3
4
5
6
7
8
9
10
11

{
   "distinct":false,
   "skip":0,
   "top":20,
   "parameters":{
   },
   "orderBy": [
   ],
   "select":[     
   ]
}

The result equals the GET operation, but now let’s update our body with a more interesting set of data:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66

{
   "distinct":false,
   "skip":0,
   "top":20,
   "parameters":{
    "Properties.prj_name": {
      "comparison": {
        "value": "test004",
        "operator": "eq"
      }
    },
    "CreatedBy$Properties.Name": {
      "comparison": {
        "value": ["awdev@awp", ""],
        "operator": "InList"
      }
    },
    "Properties.prj_prio": {
      "comparison": {
        "value": "l*",
        "operator": "Like"
      }
    }
   },
   "orderBy": [
    {
      "name": "Properties.prj_name",
      "sortType": "DESC"
    }
   ],
   "select": [
    {
      "name": "Properties.prj_name"
    },
    {
      "name": "Properties.prj_rating"
    },
    {
      "name": "Identity.Id"
    },
    {
      "name": "Title.Title"
    },
    {
      "name": "Tracking.CreatedDate"
    },
    {
      "name": "Lifecycle.CurrentState"
    },
    {
      "name": "File.MimeType"
    },
    {
      "name": "Identity.IsTemporaryCopy"
    },
    {
      "name": "CreatedBy$Properties.Name"
    },
    {
      "name": "LastModifiedBy$Properties.Name"
    },
    {
      "name": "AssigneeIdentity$Properties.IdentityDisplayName"
    }
  ]
}

Some notes from my side after playing with this data for 30 min.:

• The last 4 properties in my select statement are never retrieved; It’s strange as I can perfectly use them in my ‘parameters’ for further filtering!?
• The ‘InList’ operation is not 100% implemented as ‘[“test001”, “test004”]’ gave back all 4 of my project instances!? Adding a second ‘eq’ operation on the same property is not solving the issue as it seems to only picking up the first one.
• Adding multiple ‘parameters’ behaves as AND-logic operator
  • This is the same for the ‘orderBy’
  • Your OR-logic is found in operators like ‘InList’ and ‘Like’
• The API documentation quotes ‘IsList’, but should be ‘InList’…#SUPPORT

Does this whole exercise help me on exposing my missing properties? Well, not at all!? Strange! What we could try, but I leave that task with you, is calling my runtime service endpoint http://192.168.56.107:8080/home/appworks_tips/app/entityRestService/Elements({list_bb_id})/ResultItems with a POST request and sending the above body with it! Tell me about the outcome in the comment…I’m very curious! 😏

WAIT…There is another secret way of exposing data…Tell me, tell me!…That’s over the ReST gateway; We explained this already in this post! Have a read yourself on the possibilities as it’s (in my opinion) an underrated connector!

Answering the greatest question of all questions

We expose this data to the outside world, but how on earth can an “Outsider” call all this stuff as we only run it from within AppWorks (or in a separate tab where the SAML token for the session is already available)? Great question and great thinking…You paid attention! Have a read here for that valuable answer!

That’s a nicely covered “DONE” with interesting thoughts, assignments and explorations. We covered all kinds of ways to expose entity data to the outside world. We even made a secret call to expose even more information beyond our expectations. If you know other ways to expose data not covered in this post…Let me know in the comments!? Have a great data-exposure-weekend and I will see you next week. Cheers!

Don’t forget to subscribe to get updates on the activities happening on this site. Have you noticed the quiz where you find out if you are also “The AppWorks guy”?