A rough exploration of running Temporal.io within Nx Monorepos
Dec 27, 2021
This article assumes you’re familiar with the following concepts in Nx:
And what Temporal.io is
git clone git@github.com:andreasasprou/nx-temporal.git
cd nx-temporal
yarn
yarn nx run temporal-worker:build --prod
node dist/apps/temporal-worker/main.js
At this point, we get an error similar to:
The "path" argument must be of type string or an instance of Buffer or URL. Received type number (872)
This is down to webpack’s optimisation. So let’s disable that for now in apps/temproal-worker/project.json
:
Now, if we re-build & get an error similar to:
➜ nx-temporal git:(main) ✗ node dist/apps/temporal-worker/main.js
2021-12-18T16:24:23.104Z [INFO] [temporal_sdk_core] Registering worker task_queue="tutorial"
2021-12-18T16:24:23.624Z [ERROR] asset main.js 387 KiB [emitted] (name: main)
2021-12-18T16:24:23.624Z [ERROR] runtime modules 670 bytes 3 modules
2021-12-18T16:24:23.624Z [ERROR] modules by path ./node_modules/@temporalio/ 91.9 KiB
2021-12-18T16:24:23.624Z [ERROR] modules by path ./node_modules/@temporalio/common/lib/ 50.8 KiB
2021-12-18T16:24:23.624Z [ERROR] modules by path ./node_modules/@temporalio/common/lib/*.js 42.3 KiB 14 modules
2021-12-18T16:24:23.624Z [ERROR] modules by path ./node_modules/@temporalio/common/lib/converter/*.js 8.52 KiB
2021-12-18T16:24:23.624Z [ERROR] ./node_modules/@temporalio/common/lib/converter/data-converter.js 4.37 KiB [built] [code generated]
2021-12-18T16:24:23.624Z [ERROR] ./node_modules/@temporalio/common/lib/converter/types.js 1.22 KiB [built] [code generated]
2021-12-18T16:24:23.624Z [ERROR] ./node_modules/@temporalio/common/lib/converter/payload-converter.js 2.93 KiB [built] [code generated]
2021-12-18T16:24:23.624Z [ERROR] modules by path ./node_modules/@temporalio/workflow/lib/*.js 41.1 KiB 6 modules
2021-12-18T16:24:23.624Z [ERROR] ../../../../../src/main.js 398 bytes [built] [code generated]
2021-12-18T16:24:23.624Z [ERROR] ./node_modules/long/src/long.js 39.2 KiB [built] [code generated]
2021-12-18T16:24:23.624Z [ERROR] ./node_modules/ms/index.js 2.95 KiB [built] [code generated]
2021-12-18T16:24:23.624Z [ERROR]
2021-12-18T16:24:23.624Z [ERROR] ERROR in ../../../../../src/main.js
2021-12-18T16:24:23.624Z [ERROR] Module not found: Error: Can't resolve './apps/temporal-worker/src/workflows.ts' in '/src'
2021-12-18T16:24:23.624Z [ERROR] resolve './apps/temporal-worker/src/workflows.ts' in '/src'
2021-12-18T16:24:23.624Z [ERROR] No description file found in /src or above
2021-12-18T16:24:23.624Z [ERROR] No description file found in /src/apps/temporal-worker/src or above
2021-12-18T16:24:23.624Z [ERROR] no extension
2021-12-18T16:24:23.624Z [ERROR] /src/apps/temporal-worker/src/workflows.ts doesn't exist
2021-12-18T16:24:23.624Z [ERROR] .ts
2021-12-18T16:24:23.624Z [ERROR] /src/apps/temporal-worker/src/workflows.ts.ts doesn't exist
2021-12-18T16:24:23.624Z [ERROR] .js
2021-12-18T16:24:23.624Z [ERROR] /src/apps/temporal-worker/src/workflows.ts.js doesn't exist
2021-12-18T16:24:23.624Z [ERROR] as directory
2021-12-18T16:24:23.624Z [ERROR] /src/apps/temporal-worker/src/workflows.ts doesn't exist
2021-12-18T16:24:23.624Z [ERROR]
2021-12-18T16:24:23.624Z [ERROR] webpack 5.65.0 compiled with 1 error in 327 ms
We can investigate by looking at the built worker file where we can find the run
function:
function run() {
return (0, tslib_1.__awaiter)(this, void 0, void 0, function* () {
const worker = yield worker_1.Worker.create({
workflowsPath: /*require.resolve*/("./apps/temporal-worker/src/workflows.ts"),
activities,
taskQueue: 'tutorial',
});
yield worker.run();
});
}
Webpack is trying to bundle the workflows.ts
file and use Webpacks custom require.resolve
to retrieve the module id from the locally bundled code.
There are a few ways we can solve this, using Prebuild Workflow Bundles. I’ve explored:
workflow.ts
separately: PR ❌workflow.ts
separately: PR ❌tsc
, so we have to replace them using something like tsc-aliasThe only solution which seems to work so far is Solution 3.
If you can use esbuild in your project (i.e. you don’t need to run the Typescript compiler for features like emitDecoratorMetadata) then you can use a esbuild nx builder instead of the custom one I wrote in this PR.
Solution 1 & 2 both have issues when running the built application and starting a workflow:
-> node dist/apps/temporal-worker/main.js
2021-12-18T16:57:17.758Z [INFO] [temporal_sdk_core] Registering worker task_queue="tutorial"
2021-12-18T16:57:17.858Z [INFO] Worker state changed { state: 'RUNNING' }
2021-12-18T16:57:42.006Z [ERROR] Failed to activate workflow {
runId: '102e52a3-12e9-47fc-b048-0349ace119e0',
error: workflow-isolate:15
module.exports = require("tslib");
^
ReferenceError: require is not defined
at Object.tslib (workflow-isolate:15:1)
at __webpack_require__ (workflow-isolate:39:41)
at workflow-isolate:53:17
at workflow-isolate:66:3
at workflow-isolate:71:12
at Script.runInContext (vm.js:143:18)
Reproduction steps
git clone git@github.com:andreasasprou/nx-temporal.git
cd nx-temporal && yarn
git checkout workflows-build-step
yarn nx run temporal-worker:build --prod
node dist/apps/temporal-worker/main.js
# In another shell
ts-node apps/temporal-worker/src/client.ts
Solution 1 & 2 also have issues with the development server:
> nx run temporal-worker:serve
chunk (runtime: main) main.js (main) 2.05 KiB [entry] [rendered]
webpack compiled successfully (404e652e986324f3)
Debugger listening on ws://localhost:9229/1338b338-24f1-4e88-b6d8-abfa7faaa813
Debugger listening on ws://localhost:9229/1338b338-24f1-4e88-b6d8-abfa7faaa813
For help, see: https://nodejs.org/en/docs/inspector
Issues checking in progress...
2021-12-18T16:44:03.263Z [INFO] [temporal_sdk_core] Registering worker task_queue="tutorial"
2021-12-18T16:44:03.803Z [ERROR] asset main.js 387 KiB [emitted] (name: main)
2021-12-18T16:44:03.803Z [ERROR] runtime modules 670 bytes 3 modules
2021-12-18T16:44:03.803Z [ERROR] modules by path ./node_modules/@temporalio/ 91.9 KiB
2021-12-18T16:44:03.803Z [ERROR] modules by path ./node_modules/@temporalio/common/lib/ 50.8 KiB
2021-12-18T16:44:03.803Z [ERROR] modules by path ./node_modules/@temporalio/common/lib/*.js 42.3 KiB 14 modules
2021-12-18T16:44:03.803Z [ERROR] modules by path ./node_modules/@temporalio/common/lib/converter/*.js 8.52 KiB
2021-12-18T16:44:03.803Z [ERROR] ./node_modules/@temporalio/common/lib/converter/data-converter.js 4.37 KiB [built] [code generated]
2021-12-18T16:44:03.803Z [ERROR] ./node_modules/@temporalio/common/lib/converter/types.js 1.22 KiB [built] [code generated]
2021-12-18T16:44:03.803Z [ERROR] ./node_modules/@temporalio/common/lib/converter/payload-converter.js 2.93 KiB [built] [code generated]
2021-12-18T16:44:03.803Z [ERROR] modules by path ./node_modules/@temporalio/workflow/lib/*.js 41.1 KiB 6 modules
2021-12-18T16:44:03.803Z [ERROR] ../../../../../src/main.js 398 bytes [built] [code generated]
2021-12-18T16:44:03.803Z [ERROR] ./node_modules/long/src/long.js 39.2 KiB [built] [code generated]
2021-12-18T16:44:03.803Z [ERROR] ./node_modules/ms/index.js 2.95 KiB [built] [code generated]
2021-12-18T16:44:03.803Z [ERROR]
2021-12-18T16:44:03.803Z [ERROR] ERROR in ../../../../../src/main.js
2021-12-18T16:44:03.803Z [ERROR] Module not found: Error: Can't resolve './apps/temporal-worker/src/workflows.ts' in '/src'
2021-12-18T16:44:03.803Z [ERROR] resolve './apps/temporal-worker/src/workflows.ts' in '/src'
2021-12-18T16:44:03.803Z [ERROR] No description file found in /src or above
2021-12-18T16:44:03.803Z [ERROR] No description file found in /src/apps/temporal-worker/src or above
2021-12-18T16:44:03.803Z [ERROR] no extension
2021-12-18T16:44:03.803Z [ERROR] /src/apps/temporal-worker/src/workflows.ts doesn't exist
2021-12-18T16:44:03.803Z [ERROR] .ts
2021-12-18T16:44:03.803Z [ERROR] /src/apps/temporal-worker/src/workflows.ts.ts doesn't exist
2021-12-18T16:44:03.803Z [ERROR] .js
2021-12-18T16:44:03.803Z [ERROR] /src/apps/temporal-worker/src/workflows.ts.js doesn't exist
2021-12-18T16:44:03.803Z [ERROR] as directory
2021-12-18T16:44:03.803Z [ERROR] /src/apps/temporal-worker/src/workflows.ts doesn't exist
2021-12-18T16:44:03.803Z [ERROR]
2021-12-18T16:44:03.803Z [ERROR] webpack 5.65.0 compiled with 1 error in 329 ms
Reproduction steps
git clone git@github.com:andreasasprou/nx-temporal.git
cd nx-temporal && yarn
git checkout workflows-build-step
yarn nx run temporal-worker:serve
Get the latest from me
If you want to hear about updates about this place (new posts, new awesome products I find etc) add your email below:
If you'd like to get in touch, consider writing an email or reaching out on X.
Check this site out on Github.