Serverless deploy/package broken?

Bug Report

delta:server benlewis$ serverless -v
Framework Core: 1.67.3
Plugin: 3.6.6
SDK: 2.3.0
Components: 2.29.1

Description

Trying to deploy an existing serverless project. Running serverless deploy, the command runs for a short time (packaging via webpack), but then exits with no error/message and the function is not deployed.

Same happens when running serverless package, or deploying through CD on serverless.com. The portal reports a successful deployment, but the application is not updated (verified by downloading the function from lambda - an old version). CD logs print success, followed by undefined where there is usually a link to view deployment:

Here is the contents of the deployment bucket, note two deploys on 11th April.

And on the dashboard, 7 β€˜successful’ deployments (screenshot is truncated):

Based on the names of the folders in the deployment bucket, the last successful deploy was at 2020-04-11 11:23:28.

I’ve tried reverting to a known good commit from a week ago, and no difference.

Here is the full output of SLS_DEBUG=* serverless deploy:

delta:server benlewis$ SLS_DEBUG=* serverless deploy
Serverless: Load command interactiveCli
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command config:tabcompletion
Serverless: Load command config:tabcompletion:install
Serverless: Load command config:tabcompletion:uninstall
Serverless: Load command create
Serverless: Load command install
Serverless: Load command package
Serverless: Load command deploy
Serverless: Load command deploy:function
Serverless: Load command deploy:list
Serverless: Load command deploy:list:functions
Serverless: Load command invoke
Serverless: Load command invoke:local
Serverless: Load command info
Serverless: Load command logs
Serverless: Load command metrics
Serverless: Load command print
Serverless: Load command remove
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command slstats
Serverless: Load command plugin
Serverless: Load command plugin
Serverless: Load command plugin:install
Serverless: Load command plugin
Serverless: Load command plugin:uninstall
Serverless: Load command plugin
Serverless: Load command plugin:list
Serverless: Load command plugin
Serverless: Load command plugin:search
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command upgrade
Serverless: Load command uninstall
Serverless: Load command webpack
Serverless: Load command offline
Serverless: Load command offline:start
Serverless: Load command login
Serverless: Load command logout
Serverless: Load command generate-event
Serverless: Load command test
Serverless: Load command dashboard
Serverless: Load command output
Serverless: Load command output:get
Serverless: Load command output:list
Serverless: Load command param
Serverless: Load command param:get
Serverless: Load command param:list
Serverless: Load command dev
platform-sdk fetching: POST https://api.serverless.com/core/tenants/infstream/applications/infstream/profileValue
Serverless: Invoke deploy
Serverless: Invoke package
Serverless: Invoke aws:common:validate
Serverless: Invoke aws:common:cleanupTempDir
Serverless: Invoke webpack:validate
creating new sentry release 90516059603931c39714602fb3d6c98c0536fe40
Serverless: Invoke webpack:compile
Serverless: Bundling with Webpack...
> Analyzing 8 sources
> Rewriting sources
> Adding source map references
> Analyzing 8 sources
> Rewriting sources
> Adding source map references
> Analyzing 8 sources
> ~/mux/src/webhooks/mux/index.js
β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘ 0/8
> Analyzing 8 sources
> Rewriting sources
> Adding source map references
> Bundled 8 files for upload
> Uploading release files...
β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘  128.40KB/144.33KB (0s)
> Bundled 8 files for upload
> Uploading release files...
β ‰ Processing files...
> Bundling files for upload...
> Bundled 8 files for upload
> Uploading release files...
β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘  64.18KB/144.33KB (0s)
> Uploading release files...
> File upload complete

Source Map Upload Report
  Scripts
    ~/graphql/src/graphql/index.js
    ~/hasura/src/webhooks/hasura/index.js
    ~/mux/src/webhooks/mux/index.js
    ~/stripe/src/webhooks/stripe/index.js
  Source Maps
    ~/graphql/src/graphql/index.js.map
    ~/hasura/src/webhooks/hasura/index.js.map
> Uploaded release files to Sentry
> File upload complete

Source Map Upload Report
  Scripts
    ~/graphql/src/graphql/index.js
    ~/hasura/src/webhooks/hasura/index.js
    ~/mux/src/webhooks/mux/index.js
    ~/stripe/src/webhooks/stripe/index.js
  Source Maps
    ~/graphql/src/graphql/index.js.map
    ~/hasura/src/webhooks/hasura/index.js.map
> Uploaded release files to Sentry
> File upload complete

Source Map Upload Report
  Scripts
    ~/graphql/src/graphql/index.js
    ~/hasura/src/webhooks/hasura/index.js
    ~/mux/src/webhooks/mux/index.js
    ~/stripe/src/webhooks/stripe/index.js
  Source Maps
    ~/graphql/src/graphql/index.js.map
    ~/hasura/src/webhooks/hasura/index.js.map
    ~/mux/src/webhooks/mux/index.js.map
> File upload complete

Source Map Upload Report
  Scripts
    ~/graphql/src/graphql/index.js
    ~/hasura/src/webhooks/hasura/index.js
    ~/mux/src/webhooks/mux/index.js
    ~/stripe/src/webhooks/stripe/index.js
  Source Maps
    ~/graphql/src/graphql/index.js.map
    ~/hasura/src/webhooks/hasura/index.js.map
    ~/mux/src/webhooks/mux/index.js.map
    ~/stripe/src/webhooks/stripe/index.js.map
Time: 6295ms
Built at: 04/11/2020 4:35:40 PM
                   Asset     Size  Chunks                   Chunk Names
    src/graphql/index.js  136 KiB       0  [emitted]        src/graphql/index
src/graphql/index.js.map  219 KiB       0  [emitted] [dev]  src/graphql/index
Entrypoint src/graphql/index = src/graphql/index.js src/graphql/index.js.map
 [0] external "graphql-tag" 42 bytes {0} [built]
 [1] external "source-map-support/register" 42 bytes {0} [built]
 [4] external "apollo-errors" 42 bytes {0} [built]
 [8] external "jsonwebtoken" 42 bytes {0} [built]
[11] external "apollo-server-lambda" 42 bytes {0} [built]
[12] external "graphql-tools" 42 bytes {0} [built]
[13] ./src/graphql/schema/auth.gql 4.09 KiB {0} [built]
[20] ./src/graphql/schema/assets.gql 7.86 KiB {0} [built]
[25] ./src/graphql/schema/channels.gql 7.39 KiB {0} [built]
[28] ./src/graphql/schema/admin.gql 5.52 KiB {0} [built]
[29] ./src/graphql/schema/user.gql 3.39 KiB {0} [built]
[30] multi ./node_modules/@sentry/webpack-plugin/src/sentry-webpack.module.js ./src/graphql/index.js 40 bytes {0} [built]
[31] ./node_modules/@sentry/webpack-plugin/src/sentry-webpack.module.js 187 bytes {0} [built]
[32] external "@sentry/node" 42 bytes {0} [built]
[33] ./src/graphql/index.js + 52 modules 96 KiB {0} [built]
     | ./src/graphql/index.js 879 bytes [built]
     | ./src/shared/sentry.js 127 bytes [built]
     | ./src/shared/helpers/users.js 2.58 KiB [built]
     | ./src/graphql/schema.js 760 bytes [built]
     | ./src/shared/db/apollo.js 2.17 KiB [built]
     | ./src/shared/queries/users.js 1.42 KiB [built]
     | ./src/shared/queries/stripe.js 6.36 KiB [built]
     | ./src/graphql/resolvers/auth/index.js 477 bytes [built]
     | ./src/graphql/resolvers/assets/index.js 314 bytes [built]
     | ./src/graphql/resolvers/channels/index.js 521 bytes [built]
     | ./src/graphql/resolvers/admin/index.js 400 bytes [built]
     | ./src/graphql/resolvers/user/index.js 314 bytes [built]
     | ./src/graphql/resolvers/auth/queries/exchange-token.js 665 bytes [built]
     | ./src/graphql/resolvers/auth/queries/analytics-token.js 639 bytes [built]
     | ./src/graphql/resolvers/auth/queries/stripe-login.js 897 bytes [built]
     |     + 38 hidden modules
    + 19 hidden modules
Time: 5318ms
Built at: 04/11/2020 4:35:39 PM
                        Asset      Size  Chunks                   Chunk Names
    src/webhooks/mux/index.js  15.9 KiB       0  [emitted]        src/webhooks/mux/index
src/webhooks/mux/index.js.map  22.9 KiB       0  [emitted] [dev]  src/webhooks/mux/index
Entrypoint src/webhooks/mux/index = src/webhooks/mux/index.js src/webhooks/mux/index.js.map
 [0] external "source-map-support/register" 42 bytes {0} [built]
 [1] external "graphql-tag" 42 bytes {0} [built]
 [2] external "@mux/mux-node" 42 bytes {0} [built]
 [3] external "apollo-link" 42 bytes {0} [built]
 [4] external "@zeit/fetch" 42 bytes {0} [built]
 [5] external "apollo-client" 42 bytes {0} [built]
 [6] external "apollo-cache-inmemory" 42 bytes {0} [built]
 [7] external "apollo-link-http" 42 bytes {0} [built]
 [8] external "apollo-link-error" 42 bytes {0} [built]
 [9] external "uuid/v4" 42 bytes {0} [built]
[10] external "request" 42 bytes {0} [built]
[13] multi ./node_modules/@sentry/webpack-plugin/src/sentry-webpack.module.js ./src/webhooks/mux/index.js 40 bytes {0} [built]
[14] ./node_modules/@sentry/webpack-plugin/src/sentry-webpack.module.js 187 bytes {0} [built]
[15] external "@sentry/node" 42 bytes {0} [built]
[16] ./src/webhooks/mux/index.js + 7 modules 9.17 KiB {0} [built]
     | ./src/webhooks/mux/index.js 1.32 KiB [built]
     | ./src/shared/sentry.js 127 bytes [built]
     | ./src/webhooks/mux/events/asset.created.js 1.03 KiB [built]
     | ./src/webhooks/mux/events/asset.ready.js 719 bytes [built]
     | ./src/webhooks/mux/events/asset.deleted.js 375 bytes [built]
     | ./src/shared/db/apollo.js 2.17 KiB [built]
     | ./src/shared/queries/assets.js 2 KiB [built]
     | ./src/shared/helpers/assets.js 1.4 KiB [built]
    + 2 hidden modules
Time: 6261ms
Built at: 04/11/2020 4:35:40 PM
                           Asset      Size  Chunks                   Chunk Names
    src/webhooks/stripe/index.js  72.9 KiB       0  [emitted]        src/webhooks/stripe/index
src/webhooks/stripe/index.js.map   122 KiB       0  [emitted] [dev]  src/webhooks/stripe/index
Entrypoint src/webhooks/stripe/index = src/webhooks/stripe/index.js src/webhooks/stripe/index.js.map
 [0] external "graphql-tag" 42 bytes {0} [built]
 [1] external "source-map-support/register" 42 bytes {0} [built]
 [2] external "stripe" 42 bytes {0} [built]
 [5] external "apollo-link" 42 bytes {0} [built]
 [6] external "lodash" 42 bytes {0} [built]
 [7] external "@zeit/fetch" 42 bytes {0} [built]
 [8] external "apollo-client" 42 bytes {0} [built]
 [9] external "apollo-cache-inmemory" 42 bytes {0} [built]
[10] external "apollo-link-http" 42 bytes {0} [built]
[11] external "apollo-link-error" 42 bytes {0} [built]
[13] external "ioredis" 42 bytes {0} [built]
[15] multi ./node_modules/@sentry/webpack-plugin/src/sentry-webpack.module.js ./src/webhooks/stripe/index.js 40 bytes {0} [built]
[16] ./node_modules/@sentry/webpack-plugin/src/sentry-webpack.module.js 187 bytes {0} [built]
[17] external "@sentry/node" 42 bytes {0} [built]
[18] ./src/webhooks/stripe/index.js + 16 modules 64.9 KiB {0} [built]
     | ./src/webhooks/stripe/index.js 1.69 KiB [built]
     | ./src/shared/sentry.js 127 bytes [built]
     | ./src/shared/helpers/v2/PlatformStripeManager.js 4.59 KiB [built]
     | ./src/webhooks/stripe/events/invoice.created.js 1.63 KiB [built]
     | ./src/webhooks/stripe/events/invoice.payment_succeeded.js 1.45 KiB [built]
     | ./src/webhooks/stripe/events/invoice.payment_failed.js 126 bytes [built]
     | ./src/webhooks/stripe/events/invoice.payment_action_required.js 126 bytes [built]
     | ./src/shared/db/apollo.js 2.17 KiB [built]
     | ./src/shared/helpers/v2/InvoiceManager.js 4.2 KiB [built]
     | ./src/shared/helpers/v2/UserSubscription.js 9.93 KiB [built]
     | ./src/shared/helpers/v2/InvoiceGroup.js 2.79 KiB [built]
     | ./src/shared/helpers/v2/LockManager.js 1.88 KiB [built]
     | ./src/shared/helpers/v2/UserStripeManager.js 15.1 KiB [built]
     | ./src/shared/helpers/v2/Invoice.js 8.58 KiB [built]
     | ./src/shared/helpers/v2/InvoiceItem.js 3.15 KiB [built]
     |     + 2 hidden modules
    + 4 hidden modules
Time: 5798ms
Built at: 04/11/2020 4:35:40 PM
                           Asset      Size  Chunks                   Chunk Names
    src/webhooks/hasura/index.js  22.8 KiB       0  [emitted]        src/webhooks/hasura/index
src/webhooks/hasura/index.js.map    31 KiB       0  [emitted] [dev]  src/webhooks/hasura/index
Entrypoint src/webhooks/hasura/index = src/webhooks/hasura/index.js src/webhooks/hasura/index.js.map
 [0] external "source-map-support/register" 42 bytes {0} [built]
 [1] external "graphql-tag" 42 bytes {0} [built]
 [3] external "recombee-api-client" 42 bytes {0} [built]
 [4] external "apollo-link" 42 bytes {0} [built]
 [5] external "moment" 42 bytes {0} [built]
 [6] external "@zeit/fetch" 42 bytes {0} [built]
 [7] external "apollo-client" 42 bytes {0} [built]
 [8] external "apollo-cache-inmemory" 42 bytes {0} [built]
 [9] external "apollo-link-http" 42 bytes {0} [built]
[10] external "apollo-link-error" 42 bytes {0} [built]
[11] external "p-iteration" 42 bytes {0} [built]
[13] multi ./node_modules/@sentry/webpack-plugin/src/sentry-webpack.module.js ./src/webhooks/hasura/index.js 40 bytes {0} [built]
[14] ./node_modules/@sentry/webpack-plugin/src/sentry-webpack.module.js 187 bytes {0} [built]
[15] external "@sentry/node" 42 bytes {0} [built]
[16] ./src/webhooks/hasura/index.js + 10 modules 13.5 KiB {0} [built]
     | ./src/webhooks/hasura/index.js 1.01 KiB [built]
     | ./src/shared/sentry.js 127 bytes [built]
     | ./src/webhooks/hasura/events/videos/update.js 749 bytes [built]
     | ./src/webhooks/hasura/events/channels/update.js 398 bytes [built]
     | ./src/shared/helpers/recombee.js 4.02 KiB [built]
     | ./src/shared/helpers/fcm/ChannelVideoPush.js 512 bytes [built]
     | ./src/shared/db/apollo.js 2.17 KiB [built]
     | ./src/shared/queries/channels.js 1.42 KiB [built]
     | ./src/shared/queries/videos.js 1.65 KiB [built]
     | ./src/shared/helpers/fcm/GenericPush.js 1 KiB [built]
     | ./src/shared/db/firebase.js 423 bytes [built]
    + 2 hidden modules
Serverless: Invoke webpack:package
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module apollo-link
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module @types/node-fetch
Serverless: WARNING: Could not determine version of module node-fetch
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module apollo-link
Serverless: WARNING: Could not determine version of module @types/node-fetch
Serverless: WARNING: Could not determine version of module node-fetch
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module apollo-link
Serverless: WARNING: Could not determine version of module lodash
Serverless: WARNING: Could not determine version of module @types/node-fetch
Serverless: WARNING: Could not determine version of module node-fetch
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module apollo-link
Serverless: WARNING: Could not determine version of module @types/node-fetch
Serverless: WARNING: Could not determine version of module node-fetch
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module graphql
Serverless: Package lock found - Using locked versions
Serverless: Packing external modules: graphql-tag@^2.10.1, graphql, source-map-support@^0.5.16, stripe@^7.14.0, uuid@^3.3.3, apollo-errors@^1.9.0, @mux/mux-node@^2.5.0, firebase-admin@^8.8.0, p-iteration@^1.1.8, jsonwebtoken@^8.5.1, apollo-link, dinero.js@^1.6.0, apollo-server-lambda@^2.9.12, graphql-tools@^4.0.6, @zeit/fetch@^5.1.1, @types/node-fetch, node-fetch, apollo-client@^2.6.4, apollo-cache-inmemory@^1.6.3, apollo-link-http@^1.5.16, apollo-link-error@^1.1.12, request@^2.88.0, aws-sdk@^2.585.0, aws-s3-form@^0.3.5, ioredis@^4.16.1, redlock@^4.1.0, @sentry/node@^5.15.4, lodash, moment@^2.24.0, recombee-api-client@^2.4.1
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module apollo-link
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module @types/node-fetch
Serverless: WARNING: Could not determine version of module node-fetch
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module graphql
Serverless: WARNING: Could not determine version of module graphql
delta:server benlewis$
1 Like

Having this exact issue. Deploy command hangs after webpack. Last statements before it exits with no error are: