Support custom scopes and tooltips for FEEL expression execution context

[sbuettner]

Is your feature request related to a problem? Please describe.

FEEL expressions are supported in various ways by the properties panel. They can be applied to element templates, input- and output-mappings as well as other places. Not all FEEL expressions are evaluated by zeebe and share the scope of a process instance. This is confusing for users as certain variables are not accessible depending on the scope and the execution context where the evaluation is executed.

A scope property that allows developers to specify the scope of the FEEL expression would help to provide additional guidance for users which variables are available and what they can do with a FEEL expression for different elements (user tasks, outbound- and inbound connectors).

Describe the solution you'd like

A concept for defining a scope and visual support and explanation that guides users which variables are accessible depending on the scope of a FEEL expression. Element templates for example could define the scope where a FEEL expression will be executed in. The default could be process or zeebe. The result expression that is evaluated by the connectors runtime could define a different value that would then lead to a different pop-up/tooltip and would indicate that process variables are not accessible in that expression.

A new UX concept will be required for the FEEL expression fields that allows to provide additional information to users in which scope/context an expression is executed.

Describe alternatives you've considered

We leave the situation as is and live with a slightly confusing solution.

Additional context

https://forum.camunda.io/t/rest-connector-result-expression-cant-access-process-instance-variables/52399
https://camunda.slack.com/archives/C02JLRNQQ05/p1715953406647549

[sbuettner]

@YanaSegal FYI

[barmac]

Thanks for opening this issue. From the FEEL perspective, what you are proposing is to have a way to define the context available for a given expression.
In the engine runtime, the available contexts could be:

• process variables context (defined mostly in the element outputs)
• execution context (variables related to the process instance state, if available) - not available right now
• built-in functions context

In the connector runtime, the first two above are not available. Within the bpmn-js-element-templates project, we already don't display the detected process variables for zeebe:property and zeebe:taskHeader properties.
How is that different from your requirements?

[sbuettner]

Great question @barmac, its more a UX issue from my perspective. Even if code completion is not suggesting any variables its visually not clear that a certain FEEL expression is running in a different context and with a different scope. Hence the idea to involve @YanaSegal so we can think about a visual cue to better guide our users on their journey.

[barmac]

I agree, it's easy to miss that detail, and I have also fallen into that trap. I am looking forward to your ideas how to improve this.

[nikku]

@barmac As discussed: Consider to add your diagram, explaining scopes to this issue.

[barmac]

To visualize the scopes, I created this diagram:

connector.bpmn.txt

It shows that process variables are not available in the result expression, but they can be accessed in the input and output mapping. On the other hand, the result expression can access the connector job result which is invisible to the engine.

[sbuettner]

Thanks @barmac in addition to that we have the inbound Connectors which run outside of the scope of a process. This is usually a place where users are getting confused about the scopes as they dont have access to process variables (only exception is the correlationKey variable).

[YanaSegal]

Hi, @barmac do we have any documentation that explains this:

In the engine runtime, the available contexts could be:
process variables context (defined mostly in the element outputs)
execution context (variables related to the process instance state, if available) - not available right now
built-in functions context

How can our users understand / learn about this concept?

[barmac]

AFAIK we don't speak of contexts explicitly in the documentation, but you can read about available functions or variables, e.g. https://docs.camunda.io/docs/components/connectors/protocol/rest/#output-mapping

I think the suggestions can be the main source of knowledge given we provide them (which we don't for the response, but do for outputs and built-in functions).

[YanaSegal]

@barmac and I discussed this issue today; we started by understanding the high-level problem. we also looped @johnBgood
meeting outcome:

• We created a document to list all the issues we found + potential solutions
• @johnBgood will discuss this topic with the connectors team.
• we will come up with action points to solve the user problem for this issue + follow-up issues to address all the other issues we list in the document above.

[crobbins215]

@barmac, who owns this to move forward? I see @YanaSegal is currently assigned, with the above looking like it's gone back to @johnBgood. I want to make sure we avoid connector-only solutions. I can see this as something that would be useful to support in the core, and the connector team can use it based on the use case @sbuettner set out.

[barmac]

We are now waiting for the input from @johnBgood. Some problems can be solved in the specific connector templates or in the documentation (so by the Connectors team). Then, we can discuss what is missing in the core for the element templates to be successful.

[johnBgood]

@barmac I shared the document with the team, I think @sbuettner provided some insights, as did I.
Not sure what's expected from me now?

[crobbins215]

@crobbins215 to create cross team epic