Examples with using knex on serverless + aws lambda?

claym · January 10, 2019, 2:08pm

I’m currently having some issues getting knex to behave correctly when trying to insert data via knex.

I was wondering if anyone has a good example on getting this to work properly, and if there was any gotchas that I needed to be aware of.

welkie · January 12, 2019, 8:46am

Not Knex specific but I do have an example of connecting Lambda to a Postgres database that you might find useful: https://mattwelke.com/2019/01/06/free-tier-managed-sql-with-aws-lambda-and-heroku-postgres.html

Can you post the code that isn’t working right now?

ffxsam · April 6, 2019, 5:21pm

@claym I’d love to hear how you even got knex working with serverless. It yells at me about a lot of missing packages (mssql, mysql, oracle), which I’m not even using.

Example of my output when I try to deploy:

WARNING in ./node_modules/knex/lib/seed/index.js
133:11-24 Critical dependency: the request of a dependency is an expression
 @ ./node_modules/knex/lib/seed/index.js
 @ ./node_modules/knex/lib/util/make-knex.js
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

WARNING in ./node_modules/knex/lib/seed/index.js
82:15-75 Critical dependency: the request of a dependency is an expression
 @ ./node_modules/knex/lib/seed/index.js
 @ ./node_modules/knex/lib/util/make-knex.js
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

WARNING in ./node_modules/knex/lib/migrate/sources/fs-migrations.js
83:11-67 Critical dependency: the request of a dependency is an expression
 @ ./node_modules/knex/lib/migrate/sources/fs-migrations.js
 @ ./node_modules/knex/lib/migrate/Migrator.js
 @ ./node_modules/knex/lib/util/make-knex.js
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/knex/lib/dialects/mssql/index.js
Module not found: Error: Can't resolve 'mssql' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/knex/lib/dialects/mssql'
 @ ./node_modules/knex/lib/dialects/mssql/index.js 75:25-41
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/knex/lib/dialects/mssql/index.js
Module not found: Error: Can't resolve 'mssql/lib/base' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/knex/lib/dialects/mssql'
 @ ./node_modules/knex/lib/dialects/mssql/index.js 77:17-42
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/knex/lib/dialects/mssql/index.js
Module not found: Error: Can't resolve 'mssql/package.json' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/knex/lib/dialects/mssql'
 @ ./node_modules/knex/lib/dialects/mssql/index.js 90:25-54
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/knex/lib/dialects/mysql/index.js
Module not found: Error: Can't resolve 'mysql' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/knex/lib/dialects/mysql'
 @ ./node_modules/knex/lib/dialects/mysql/index.js 45:11-27
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/knex/lib/dialects/mysql2/index.js
Module not found: Error: Can't resolve 'mysql2' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/knex/lib/dialects/mysql2'
 @ ./node_modules/knex/lib/dialects/mysql2/index.js 37:11-28
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/knex/lib/dialects/oracle/index.js
Module not found: Error: Can't resolve 'oracle' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/knex/lib/dialects/oracle'
 @ ./node_modules/knex/lib/dialects/oracle/index.js 51:11-28
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/knex/lib/dialects/oracledb/index.js
Module not found: Error: Can't resolve 'oracledb' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/knex/lib/dialects/oracledb'
 @ ./node_modules/knex/lib/dialects/oracledb/index.js 42:19-38 382:19-38
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/pg/lib/native/client.js
Module not found: Error: Can't resolve 'pg-native' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/pg/lib/native'
 @ ./node_modules/pg/lib/native/client.js 10:13-33
 @ ./node_modules/pg/lib/native/index.js
 @ ./node_modules/pg/lib/index.js
 @ ./node_modules/knex/lib/dialects/redshift/index.js
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/knex/lib/dialects/postgres/index.js
Module not found: Error: Can't resolve 'pg-query-stream' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/knex/lib/dialects/postgres'
 @ ./node_modules/knex/lib/dialects/postgres/index.js 219:56-82
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/knex/lib/dialects/sqlite3/index.js
Module not found: Error: Can't resolve 'sqlite3' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/knex/lib/dialects/sqlite3'
 @ ./node_modules/knex/lib/dialects/sqlite3/index.js 44:11-29
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

ERROR in ./node_modules/knex/lib/dialects/mssql/index.js
Module not found: Error: Can't resolve 'tedious' in '/Volumes/SuperData/Sites/reelcrafter/reelcrafter-backend/node_modules/knex/lib/dialects/mssql'
 @ ./node_modules/knex/lib/dialects/mssql/index.js 73:16-34
 @ ./node_modules/knex/lib/dialects ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./src/rds.js

lukebyrne · April 28, 2019, 11:43am

@ffxsam Did you ever find a solution to this, I am having problems and have setup this dummy repo

and am trying to get some debug support from ‘serverless-webpack’

github.com/serverless-heaven/serverless-webpack

V3: Unable to import module 'handler': Error

opened 01:09PM - 11 Oct 16 UTC

closed 01:55PM - 15 Aug 17 UTC

thierryskoda

help wanted documentation

I often have this error and I never know where to look at to fix the issue. Is t…here a way/tool to have more information about the error ? I have the `SLS_DEBUG` set to true but it doesn't help. ``` Unable to import module 'handler': Error at Function.Module._load (module.js:276:25) at Module.require (module.js:353:17) at require (internal/module.js:12:17) at Object.<anonymous> (/var/task/handler.js:373:19) at __webpack_require__ (/var/task/handler.js:20:30) at Object.<anonymous> (/var/task/handler.js:53:19) at __webpack_require__ (/var/task/handler.js:20:30) at /var/task/handler.js:40:18 at Object.<anonymous> (/var/task/handler.js:43:10) ``` Thank you very much! Thierry

matuella · June 3, 2019, 3:46pm

I was trying to fix it, but I gave up and just went full JS

CheapeOne · August 3, 2019, 4:46am

To fix this problem, I had to list knex as an external in my webpack config:

// webpack.config.js

module.exports = {
...
externals: { knex: "commonjs knex" }
}

then I had to add custom force include in my serverless config, so it doesn’t try to bundle knex but uses the one installed in the lambda’s environment?

# serverless.yml

custom:
  webpack:
    includeModules:
      forceInclude:
        - knex
        - pg # I'm using postgres

Topic		Replies	Views
Express + Knex + RDS (postgres) = Internal server error? Serverless Framework aws	3	1434	January 3, 2019
Real world example Serverless Framework aws	19	18361	November 10, 2016
Lambda vs MySql Serverless Framework aws , lambda	4	444	August 25, 2020
Use sequelize.js ORM library with servereless Serverless Framework aws	4	8930	November 9, 2018
When packaging a typescript lambda I get my dev dependencies included Serverless Framework	1	1163	November 23, 2017

Examples with using knex on serverless + aws lambda?

Related topics