[AWS] Deploy GeoDjango to AWS Lambda

使用 Zappa

$ zappa update dev
Calling update for stage dev..
Downloading and installing dependencies..
 - ruamel.yaml.clib==0.2.4: Using locally cached manylinux wheel
 - pyyaml==5.4.1: Downloading
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 637k/637k [00:00<00:00, 3.62MB/s]
 - psycopg2==2.7.7: Using locally cached manylinux wheel
 - psycopg2-binary==2.8.6: Using locally cached manylinux wheel
 - pillow==8.1.0: Downloading
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2.23M/2.23M [00:00<00:00, 4.57MB/s]
 - opencv-python==4.1.0.25: Using locally cached manylinux wheel
 - numpy==1.16.4: Using locally cached manylinux wheel
 - markupsafe==1.1.1: Downloading
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 27.5k/27.5k [00:00<00:00, 11.1MB/s]
Packaging project as zip.


Warning: Application zip package is likely to be too large for AWS Lambda. Try setting "slim_handler" to true in your Zappa settings file.


Uploading django-project-dev-1625807826.zip (97.6MiB)..
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 102M/102M [00:22<00:00, 4.61MB/s]
Updating Lambda function code..
Oh no! An error occurred! :(

==============

Traceback (most recent call last):
  File "/home/jeff/.virtualenvs/django2/lib/python3.7/site-packages/zappa/cli.py", line 3422, in handle
    sys.exit(cli.handle())
  File "/home/jeff/.virtualenvs/django2/lib/python3.7/site-packages/zappa/cli.py", line 588, in handle
    self.dispatch_command(self.command, stage)
  File "/home/jeff/.virtualenvs/django2/lib/python3.7/site-packages/zappa/cli.py", line 644, in dispatch_command
    self.vargs["docker_image_uri"],
  File "/home/jeff/.virtualenvs/django2/lib/python3.7/site-packages/zappa/cli.py", line 1158, in update
    self.lambda_arn = self.zappa.update_lambda_function(**kwargs)
  File "/home/jeff/.virtualenvs/django2/lib/python3.7/site-packages/zappa/core.py", line 1278, in update_lambda_function
    response = self.lambda_client.update_function_code(**kwargs)
  File "/home/jeff/.virtualenvs/django2/lib/python3.7/site-packages/botocore/client.py", line 386, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/home/jeff/.virtualenvs/django2/lib/python3.7/site-packages/botocore/client.py", line 705, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.InvalidParameterValueException: An error occurred (InvalidParameterValueException) when calling the UpdateFunctionCode operation: Unzipped size must be smaller than 262144000 bytes

==============

Need help? Found a bug? Let us know! :D
File bug reports on GitHub here: https://github.com/Zappa/Zappa
And join our Slack channel here: https://zappateam.slack.com
Love!,
 ~ Team Zappa!

添加 slim_handler，可能再安装：pip install supervisor==4.0.0。

Downloading and installing dependencies..
Packaging project as zip.
Uploading django-project-dev-1625808445.tar.gz (97.0MiB)..
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 102M/102M [00:17<00:00, 5.73MB/s]
Uploading handler_django-project-dev-1625808494.zip (14.4MiB)..
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 15.1M/15.1M [00:07<00:00, 1.94MB/s]
Updating Lambda function code..
Updating Lambda function configuration..
Oh no! An error occurred! :(

==============

Traceback (most recent call last):
  File "/home/jeff/.virtualenvs/django2/lib/python3.7/site-packages/zappa/cli.py", line 3422, in handle
    sys.exit(cli.handle())
  File "/home/jeff/.virtualenvs/django2/lib/python3.7/site-packages/zappa/cli.py", line 588, in handle
    self.dispatch_command(self.command, stage)
  File "/home/jeff/.virtualenvs/django2/lib/python3.7/site-packages/zappa/cli.py", line 644, in dispatch_command
    self.vargs["docker_image_uri"],
  File "/home/jeff/.virtualenvs/django2/lib/python3.7/site-packages/zappa/cli.py", line 1194, in update
    endpoint_configuration=self.endpoint_configuration,
  File "/home/jeff/.virtualenvs/django2/lib/python3.7/site-packages/zappa/core.py", line 2417, in create_stack_template
    self.cf_template.add_description("Automatically generated with Zappa")
AttributeError: 'Template' object has no attribute 'add_description'

==============

问题

Deploying API Gateway.
Error: Warning! Status check on the deployed lambda failed. A GET request to '/' yielded a 502 response code.

方案

Ref: Django deploy - Zappa onto AWS Lambda + API Gateway, 23:30/30:23

这是sqlite数据库的问题，直接在setting中注释掉相关部分就好。

$ zappa tail
Important! A new version of Zappa is available!
Upgrade with: pip install zappa --upgrade
Visit the project page on GitHub to see the latest changes: https://github.com/Miserlou/Zappa
Calling tail for stage dev..
[1625883281352] Instancing..
[1625883284695] [DEBUG] 2021-07-10T02:14:44.694Z 7eb98e50-58db-40af-aeb2-c15821d57278 Using selector: EpollSelector
[1625883285113] SQLite 3.8.3 or later is required (found 3.7.17).: ImproperlyConfigured
Traceback (most recent call last):
  File "/var/task/handler.py", line 580, in lambda_handler
  return LambdaHandler.lambda_handler(event, context)
  File "/var/task/handler.py", line 245, in lambda_handler

Ref: GeoDjango Installation

Spatial database

PostgreSQL (with PostGIS), MySQL (mostly with MyISAM engine), Oracle, and SQLite (with SpatiaLite) are the spatial databases currently supported.

RDS Database for Serverless Django + Zappa on AWS Lambda

November 9, 2018

Django with GeoDjango on AWS EC2, Postgres Postgis on RDS

Working with the PostGIS extension

zappa.json的使用

Serverless Deployment of a Django Project with AWS Lambda, Zappa, S3 and PostgreSQL

使用前，先把这个配置好。

How to Deploy a django app into aws with RDS - Part 3

第一步，配置云端数据库，并能本地链接查看。

Ref: AWS RDS PostgreSQL | Create and Locally connect with PgAdmin, DBeaver

如果是公网可访问，必须在有网关的VPC中。

第二步，对应的在settings中填写如下。

第三步，Zappa部署时，如何同时涉及到 VPC and RDS。

RDS Database for Serverless Django + Zappa on AWS Lambda【先测试这个，认认真真走一遍】

November 9, 2018

Complete Prepare AWS IAM User, Role, and Policies for Zappa and Serverless Python

先自己创立一个policy 【ZappaRolePolicy】，再创建一个role。

然后再创建一个policy【ZappaUserGeneralPolicy】，并用上一个role arn替换掉其中的部分。

之后是设置s3，类似的流程。

Complete Serverless Django with Zappa on AWS Lambda

第一次zappa上传，得到url后，记得修改ALLOWED_HOSTS。

遇到了如下问题

$ zappa deploy dev
Important! A new version of Zappa is available!
Upgrade with: pip install zappa --upgrade
Visit the project page on GitHub to see the latest changes: https://github.com/Miserlou/Zappa
Calling deploy for stage dev..
Downloading and installing dependencies..
- ruamel.yaml.clib==0.2.6: Using locally cached manylinux wheel
- psycopg2-binary==2.8.6: Using locally cached manylinux wheel
- pillow==4.3.0: Using locally cached manylinux wheel
- markupsafe==2.0.1: Using locally cached manylinux wheel
- sqlite==python36: Using precompiled lambda package
Packaging project as zip.

Warning: Application zip package is likely to be too large for AWS Lambda. Try setting "slim_handler" to true in your Zappa settings file.

Uploading test-dev-1626012189.zip (50.4MiB)..
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 52.8M/52.8M [00:29<00:00, 1.22MB/s]
Scheduling..
Scheduled test-dev-zappa-keep-warm-handler.keep_warm_callback with expression rate(4 minutes)!
Uploading test-dev-template-1626012236.json (1.5KiB)..
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.54K/1.54K [00:00<00:00, 2.60KB/s]
Waiting for stack test-dev to create (this can take a bit)..
75%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████ | 3/4 [00:10<00:05, 5.63s/res]
Deploying API Gateway..
Error: Warning! Status check on the deployed lambda failed. A GET request to '/' yielded a 502 response code.

或许还有更好的选择，比如Terraform。

Ref: Deploying a Django project on AWS Lambda using Serverless

在没有gis的情况下，先实操一遍。

Select Yes for Public access

这个例子，竟然需要打开public access。

竟然还有这种书：

《Building Serverless Python Web Services with Zappa》

By Abdulwahid Abdulhaque Barguzar

实战笔记

Ref: AWS hands-on: running serverless Django with Zappa【一步步做】

因为升级，导致的code bug，而出现下列问题。

Why do I get "AttributeError: 'Template' object has no attribute 'add_description'" error when deploy with Zappa?

Ref: https://githubmemory.com/repo/zappa/Zappa/activity?page=4

方案：The fact is that we should use set_description.

Ref: Using Zappa and AWS Lambda to deploy serverless Django apps, January 12, 2021

可能跟Django版本有关，2.1.9貌似安稳一些。

第一个：Serverless Django: Exploring the State of the Art on AWS

EC2 instances in a VPC can be assigned ephemeral public IP addresses, making them easy to wire up to the Internet. To get the same thing with a Lambda function in a VPC, you need a few more AWS resources: you need a private subnet — preferably 2 or more — that routes to a NAT Gateway with its own Elastic IP address, which forwards packets to a public subnet connected to an Internet Gateway. No resources can be removed from that stack. The NAT Gateway assigns a public IP address to packets, but can’t connect to the Internet on its own; the Internet Gateway connects a subnet to the Internet, but can’t assign a public IP address to packets on its own. The correct VPC structure for Lambda is less complicated than the official AWS documentation makes it seem.

第二个：Deploying Serverless Django with Zeit and RDS Postgres

貌似比较合适？

可能会遇到sqlite的版本问题，使用Django2.1.9即可，或者升级sqlite版本。记得修改project name，换个新的，再deploy。

Error loading MySQLdb module.

django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module.
Did you install mysqlclient?

A serverless web application using Python, API Gateway and Lambda function

Creating a REST application using AWS Chalice and Peewee

考虑使用aws自带的python工程部署方案: AWS Chalice。

太艰苦了，直接开ec2搭建数据库服务器，简单一些。

Ref: https://remotely360.com/blog/geospatial-location-intelligence/geoserver-postgis-aws

posted @ 2021-04-04 07:03 郝壹贰叁阅读(177) 评论(0) 收藏举报

刷新页面返回顶部

机器学习水很深

We all have two lives. The second one starts when we realize that we only have one. --- Tom Hiddleston

[AWS] Deploy GeoDjango to AWS Lambda

使用 Zappa

问题

方案

Spatial database

遇到了如下问题

《Building Serverless Python Web Services with Zappa》

实战笔记

公告