This isn’t a serverless specific question, but i suspect this right audience to chime in…
I was wondering if non-blocking I/O matters much less(not at all?) in a serverless world?
My feeling is that it really doesn’t matter. However, i’m basing this on the my understanding that the lambda/azure/google functions service one and only one request at a time. I’m wondering if its safe to assume this will always be true.
If so, from a programming model, doesn’t that imply that any time spent waiting for I/O, can never be used to service another request, because you’ll never get one routed to you until you exit?
I can only think that if there were some way to spawn child threads doing I/O inside your function, you’d benefit from using the wait time. I know spawning threads in the JVM is doable, pretty sure it is in python, and i think it’s impossible in node, but not sure.
At the moment non-blocking is irrelevant because as you say it only serves one request. In the future if AWS implements request parallelism inside of one lambda that might be different (just to make sure I have no knowledge of them working on it, could just be something they might want to do at some point).
So its good to assume at the moment and my assumption is that FaaS providers will make you specifically say you want to parallelise requests in one function in the future. I’d be surprised if they do it out of the box.
yeah, it would be interesting to see the result of allowing multiple concurrent requests for a single lambda instance, they would be competing for CPU/RAM/IO resources…
Just another little thought experiment…
If your use case is such that your lambda is pretty much I/O bound (like the “typical” WebApp usually) you might be getting “double billed” while your function just sits waiting for data to come back.
Example.
lambda function X spends 5ms processing input
lambda function X spends 500ms waiting for data + external I/O charges (dynamoDB or something else)
lambda function X spends 5ms prepping output
if you were a giant (netflix, facebook size) you’d prob save a ton of money if you
could have I/O services offer async webhook style responses & somehow websocket stream back to clients.
I’d love to see a Websocket endpoint on APIG where you can send back data packets similar to services like Pusher. Then the whole system on Lambda could be completely event based.
I’d assume that they would implement something like parallelism on functions, but it could also lead to massive problems and basically you’re getting closer to actual servers then. For only optimising specific things that could be interesting.