Examples with using knex on serverless + aws lambda?

I’m currently having some issues getting knex to behave correctly when trying to insert data via knex.

I was wondering if anyone has a good example on getting this to work properly, and if there was any gotchas that I needed to be aware of.

Not Knex specific but I do have an example of connecting Lambda to a Postgres database that you might find useful: https://mattwelke.com/2019/01/06/free-tier-managed-sql-with-aws-lambda-and-heroku-postgres.html

Can you post the code that isn’t working right now?

1 Like

@claym I’d love to hear how you even got knex working with serverless. It yells at me about a lot of missing packages (mssql, mysql, oracle), which I’m not even using.

Example of my output when I try to deploy:

WARNING in ./node_modules/knex/lib/seed/index.js
133:11-24 Critical dependency: the request of a dependency is an expression
 @ ./node_modules/knex/lib/seed/index.js
 @ ./node_modules/knex/lib/util/make-knex.js
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

WARNING in ./node_modules/knex/lib/seed/index.js
82:15-75 Critical dependency: the request of a dependency is an expression
 @ ./node_modules/knex/lib/seed/index.js
 @ ./node_modules/knex/lib/util/make-knex.js
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

WARNING in ./node_modules/knex/lib/migrate/sources/fs-migrations.js
83:11-67 Critical dependency: the request of a dependency is an expression
 @ ./node_modules/knex/lib/migrate/sources/fs-migrations.js
 @ ./node_modules/knex/lib/migrate/Migrator.js
 @ ./node_modules/knex/lib/util/make-knex.js
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/knex/lib/dialects/mssql/index.js
Module not found: Error: Can't resolve 'mssql' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/knex/lib/dialects/mssql'
 @ ./node_modules/knex/lib/dialects/mssql/index.js 75:25-41
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/knex/lib/dialects/mssql/index.js
Module not found: Error: Can't resolve 'mssql/lib/base' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/knex/lib/dialects/mssql'
 @ ./node_modules/knex/lib/dialects/mssql/index.js 77:17-42
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/knex/lib/dialects/mssql/index.js
Module not found: Error: Can't resolve 'mssql/package.json' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/knex/lib/dialects/mssql'
 @ ./node_modules/knex/lib/dialects/mssql/index.js 90:25-54
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/knex/lib/dialects/mysql/index.js
Module not found: Error: Can't resolve 'mysql' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/knex/lib/dialects/mysql'
 @ ./node_modules/knex/lib/dialects/mysql/index.js 45:11-27
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/knex/lib/dialects/mysql2/index.js
Module not found: Error: Can't resolve 'mysql2' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/knex/lib/dialects/mysql2'
 @ ./node_modules/knex/lib/dialects/mysql2/index.js 37:11-28
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/knex/lib/dialects/oracle/index.js
Module not found: Error: Can't resolve 'oracle' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/knex/lib/dialects/oracle'
 @ ./node_modules/knex/lib/dialects/oracle/index.js 51:11-28
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/knex/lib/dialects/oracledb/index.js
Module not found: Error: Can't resolve 'oracledb' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/knex/lib/dialects/oracledb'
 @ ./node_modules/knex/lib/dialects/oracledb/index.js 42:19-38 382:19-38
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/pg/lib/native/client.js
Module not found: Error: Can't resolve 'pg-native' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/pg/lib/native'
 @ ./node_modules/pg/lib/native/client.js 10:13-33
 @ ./node_modules/pg/lib/native/index.js
 @ ./node_modules/pg/lib/index.js
 @ ./node_modules/knex/lib/dialects/redshift/index.js
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/knex/lib/dialects/postgres/index.js
Module not found: Error: Can't resolve 'pg-query-stream' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/knex/lib/dialects/postgres'
 @ ./node_modules/knex/lib/dialects/postgres/index.js 219:56-82
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/knex/lib/dialects/sqlite3/index.js
Module not found: Error: Can't resolve 'sqlite3' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/knex/lib/dialects/sqlite3'
 @ ./node_modules/knex/lib/dialects/sqlite3/index.js 44:11-29
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/knex/lib/dialects/mssql/index.js
Module not found: Error: Can't resolve 'tedious' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/knex/lib/dialects/mssql'
 @ ./node_modules/knex/lib/dialects/mssql/index.js 73:16-34
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

@ffxsam Did you ever find a solution to this, I am having problems and have setup this dummy repo

and am trying to get some debug support from ‘serverless-webpack’

I was trying to fix it, but I gave up and just went full JS :confused:

To fix this problem, I had to list knex as an external in my webpack config:

// webpack.config.js

module.exports = {
...
externals: { knex: "commonjs knex" }
}

then I had to add custom force include in my serverless config, so it doesn’t try to bundle knex but uses the one installed in the lambda’s environment?

# serverless.yml

custom:
  webpack:
    includeModules:
      forceInclude:
        - knex
        - pg # I'm using postgres