[CLI] Ref Resolving is Failing

[ckoegel]

We use the prism CLI to unit test our SDKs, and over the last week, these unit tests have been failing due to response errors that seem to come from $refs being incorrectly resolved in the OpenAPI spec. This was previously working and no code in the tests or the spec file have been made.

Context

This bug has caused our unit test suites to fail for our SDKs across Ruby, Python, and Node.

Current Behavior

When running our test suite, one endpoint returns a 500 error due to an invalid reference token. Prior to today (8/2), there were 3 endpoints that had issues, all with similar reference issues, but as of this morning, it is now just the one. The current error is below.
[HTTP SERVER] get /accounts/***/calls/c-1234/recordings/r-1234 ✖ error Request terminated with error: Error: Invalid reference token: __bundled__

Here is a link to a run in Github where the tests succeeded 3 weeks ago and then failed on a re-run this morning.
Success: https://github.com/Bandwidth/python-sdk/actions/runs/9911218308/attempts/1
Fail: https://github.com/Bandwidth/python-sdk/actions/runs/9911218308/attempts/2

This run in Node from 2 days ago shows the previous 3 errors, all of which seem to be related.
https://github.com/Bandwidth/node-sdk/actions/runs/10187817666

Expected Behavior

These test should all pass and the $ref's should resolve properly.

Possible Workaround/Solution

I've attempted to downgrade to an old version of prism (5.8.1), but this has not helped. I'm unaware of a workaround for this.

Steps to Reproduce

Relevant GH Run Links:
https://github.com/Bandwidth/node-sdk/actions/runs/10186773988
https://github.com/Bandwidth/python-sdk/actions/runs/9911218308

1. Clone https://github.com/Bandwidth/node-sdk
2. Install dependencies

yarn
# or npm install

3. Install prism-cli

yarn global add @stoplight/[email protected]
# or npm install -g @stoplight/[email protected]

4. Set Up Environment Variables

export BASE_CALLBACK_URL=https://test.com
export BW_ACCOUNT_ID=1234567
export BW_MESSAGING_APPLICATION_ID=abcd1234-4d87-4f20-94e9-811e3b6bc673
export BW_NUMBER=+19195551234
export BW_PASSWORD=password
export BW_USERNAME=username
export BW_VOICE_APPLICATION_ID=abcd1234-46f8-4426-a9b1-af52aa723823
export MANTECA_BASE_URL=https://test.com
export MANTECA_APPLICATION_ID=abcd1234-540a-4709-a5ae-b26d3b74d55d
export USER_NUMBER=+19195554321

5. Start Prism mock server

prism mock -v debug bandwidth.yml

6. Run Test Suite

yarn test-unit
# or npm run test-unit

Environment

• Version used: 5.8.3, 5.8.1
• Environment name and version (e.g. Chrome 39, node.js 5.4):
  • Nodejs v18.19.0, 18.19.1, 18.20.1, 20.15.1, 22.5.1
  • npm v10.2.3, 10.2.4, 10.7.0, 10.8.2
  • yarn v1.22.19, 1.22.22
• Operating System and version (desktop or mobile): Windows 2019, Windows 2022, Ubuntu 20.04, Ubuntu 22.04, MacOS 14
• Link to your environment/workspace/project:
  • https://github.com/Bandwidth/ruby-sdk
  • https://github.com/Bandwidth/node-sdk
  • https://github.com/Bandwidth/python-sdk

[Didel]

I think I have experienced the same problem. Could you check if the workaround from stoplightio/json#146 fixes it for you?

[ckoegel]

@Didel
That workaround does work if I configure a package.json to override the peer dependency.
Sadly for our use case, we install the prism cli globally in a GitHub workflow using npm install -g @stoplight/prism-cli and can't override the peer dependency this way. Is there a timeline for a long-term fix for this to be released?

[Didel]

Sadly for our use case, we install the prism cli globally in a GitHub workflow using npm install -g @stoplight/prism-cli and can't override the peer dependency this way.

Hhm. Perhaps you could try to add @stoplight/json 3.21.0 as a direct dependency, that might work for now (but is less preferred than the override).

Is there a timeline for a long-term fix for this to be released?

I have no clue, I'm just a user experiencing the issue just like you 🙂

[mnaumanali94]

@ckoegel @Didel Would you be able to provide us a specification that we can use to reproduce this?

[Didel]

@ckoegel @Didel Would you be able to provide us a specification that we can use to reproduce this?

@mnaumanali94 I think the problem can be reproduced with the provided yml in stoplightio/json#146

#Behavior
The error occurs when there are multiple endpoints that respond with the same schema

That is also the case for me. The first 2 endpoints work fine, the third one throws the error mentioned in that issue. When only swapping the order of the endpoints, again, the first 2 work, the third one does not work (while the exact same configuration worked before changing the order)

[mnaumanali94]

@Didel We were trying to reproduce this using the spec in stoplightio/json#146 but struggling to get the same behavior. If you have a spec handy that we could use to reproduce this, that would be great otherwise we'll keep trying on our side.

[Didel]

@mnaumanali94 Sure! I couldn't copy/paste what I was working with, but I managed to recreate a minimal example that has the same structure and shows the same problem, see https://gist.github.com/Didel/d489c4439d0d6ea48d1cba5955e21c24

I am running NodeJS version 20.17.0 (the current latest LTS-version).

To reproduce the issue:

• Checkout above gist and/or copy files locally
• run npm install to install the only dependency given, @stoplight/prism-cli (in my case it installed version 5.9.0)
• run npm run start to start running the prism CLI with the provided spec.yml file
• This will expose three endpoints (all with the exact same data-structure):
  • http://127.0.0.1:4010/foo/bar
  • http://127.0.0.1:4010/foo/baz
  • http://127.0.0.1:4010/foo/qux
• Visiting /foo/bar and /foo/baz should work as expected and return example data.
• Visiting /foo/qux will result in the above error.
• Only changing the order of the paths in the yml-file will change the behaviour: The first two defined endpoints will work, the last one will result in above error. Edit: After some testing and adding more endpoints, it appears that only the first two defined endpoints will keep working, the third and any other consecutive endpoints will be broken.
• Applying the work-around to the package.json file (adding the override) and running npm install and npm run start again afterwards, without any changes to the spec.yml file, will result in the error not occurring any more.

I hope this should provide enough information to reproduce the error!

[zackbraksa]

@ckoegel Another possible workaround would be to generate a dereferenced spec using swagger-cli and calling prism mock on the dereferenced/resolved file, for example:

swagger-cli bundle -r -o bandwidth.resolved.yml bandwidth.yml
prism mock bandwidth.resolved.yml

[Didel]

@mnaumanali94 Sure! I couldn't copy/paste what I was working with, but I managed to recreate a minimal example that has the same structure and shows the same problem, see https://gist.github.com/Didel/d489c4439d0d6ea48d1cba5955e21c24
!

@mnaumanali94 Were you able to reproduce the issue with the above gist?

[Didel]

It seems that this issue is resolved in the current version of Prism (5.12.0 and higher). The above gist also doesn't show the problem any more, so this issue can likely be closed.