I have a quite large tensorflow model I want to evaluate and serve. The model itself is about 200 MB and my dependencies are also roughly 200 MB. How would I even go about loading this into lambda (since there is an upper limit of 250 MB unzipped). Should I just create an s3 bucket containing these files and fetch them at the top of my handle function?
I believe your use case doesn’t fit well for Lambda. Try running ur functionality in a nodejs app on aws beanstalk.
How do you mean “evaluate”? And when you say “serve” do you mean the results of that evaulation or do you mean the original file?
The abstract answer to how to do something that requires a lot of computation or a lot of memory is to attempt to break the problem up into smaller chunks and then process those smaller chunks instead. The specifics are where things get tricky.
So I don’t know much about tensorflow but I found this in their docs:
https://www.tensorflow.org/deploy/distributed
Perhaps you have a distributed system running and then you are simply controlling or calling into that system from your lambda.
There may be some other way to implement the processing as a database and a series of lambdas but it could be that it is just simply not designed in a way that will be possible without a ton of work.
I think I was unclear about the terminology but basically I already have my model and can get a prediction out of it by feeding it an Image URL (it takes care of everything) and then spits out an prediction of whats in that image. But I need to have dependencies for that model and the model itself loaded into the filesystem (which presents an issue since serverless and lambda won’t let me upload a bundle > 260 MB in size). I am just looking for a way to perhaps download the needed model and dependencies into the lambda “filesystem” on instantiation. Would a valid solution be to upload my model and dependencies to an s3 bucket and, when my lambda function runs, download that stuff into /tmp ?
Lambda also has a memory limitation. You can crank up the memory to just over 1GB I think but if processing the file takes more than that then that could be another problem.
But if that isn’t a problem and it fits in those memory constraints then you definitely can put it into s3 and then load the file from there. The only problem with this is that it will read it from s3 upon every invocation of the lambda. Depending on your usage patterns it could get expensive. Is this something you’re going to be doing multiple times a second? Or more like multiple times a day?
It’ll be close to a few times an hour. If I keep the lambda instance warm shouldn’t it not need to redownload from S3 every time?
Maybe, but at a few times an hour downloading shouldn’t be a problem. It should only take a second or two at that size too. Very fast.