使用Backblaze B2+Cloudflare+rclone为小盘机(免费)增加空间

小盘机的空间经常不够用:我们可以把Backblaze B210G免费空间搬到小鸡上。

你需要:

  • 接码手机号和邮箱
  • (可选)一个顶级域名,供注册Cloudflare

技术

Rclone

Rclone最大的作用是同步文件。Rclone支持很多稀奇古怪的数据源,以及很多稀奇古怪的功能 – 其中包括将远程文件挂载成FUSE文件系统。

利用类似本文的方法,也可以将Google Drive,Onedrive,Dropbox,远程大盘鸡等挂载成文件系统。

Backblaze B2

Backblaze的服务器在美国西部,虽然只有一个机房但是便宜:每T存储每月5.12 USD。

为什么要白嫖B2而不是其他的存储?

  1. Backblaze加入了Cloudflare的Bandwidth Alliance,下载流量免费;上传不收费。

  2. Backblaze B2是企业级存储:8个9的可靠性,3个9的可用性。不会像个人网盘删文件或限制API请求频率。

  3. 每个账户送10G存储空间,每日1G下载流量,2500次下载请求。对于白嫖够用了。

设置

Backblaze B2

注册Backblaze B2

点击 https://www.backblaze.com/b2/sign-up.html 注册。

注册后使用邮箱和密码登陆:第一次登陆需要绑定手机号。请自由发挥。+86应该可以使用。

创建仓库

在https://secure.backblaze.com/b2_buckets.htm 处,点击“Create a Bucket”。

  • Bucket Unique Name: 仓库的名字。必须全局唯一,仅限字母和数字。
  • Files in Bucket are: 公开还是私密仓库。私密(private)即可,除非你不在乎公开文件。
  • Object Lock: 锁定文件一段时间内不能删除。不需要开启。

记录仓库的名字。点击蓝色的Create a Bucket 创建仓库。

和所有的对象存储一样,B2默认保存文件的所有版本:如果要修改,点击仓库的Lifecycle Settings

  • Keep all versions of the file (default):默认保存所有版本
  • Keep only the last version of the file:只保存最后一个版本。我会使用这个节省空间。
  • Keep prior versions for this number of days:在X天内保存旧版本。
  • Use custom lifecycle rules:按文件前缀自定义多少天隐藏,多少天删除文件。

创建APP Key

默认的key不能使用:在https://secure.backblaze.com/app_keys.htm 创建一个新key。

点击Add a New Application Key

  • Name of Key: 名字,供你自己参考
  • Allow access to Bucket(s): 你可以让key只能访问某个仓库。
  • Type of Access: 只读,只写或读写都可以。除非你知道你在做什么,否则选择Read and Write读写均可。
  • File name prefix: 只能访问前缀为此项的文件。除非你知道你在做什么,否则留空。
  • Duration (seconds): 有效期。除非你知道你在做什么,否则留空。

点击Create New Key。你会看见创建的key:立即保存,这些信息只出现一次。

记录keyIDapplicationKey备用。

数据端点

首先找到你的数据端点:在https://tree-sac0-0001.backblaze.com/b2_browse_files2.htm ,点击你创建的仓库;

点击Upload,上传一个文件 – 哪怕只有1字节也行。上传成功后,点击这个文件。

你会看见很多链接:找到Friendly URL,看域名是什么。例如https://f002.backblazeb2.com/file/bntest/test.tgz代表 f002.backblazeb2.com

记录这个端点值。

Cloudflare

我用自己的域名(2012年至今,应该永久续费)和CF Pro做了如下设置:

这几个记录都开启了Cloudflare转发。

如果你想用自己的域名,按下面的流程注册Cloudflare并绑定域名,设置CNAME:如果懒得搞,用我提供的域名即可 – 从上文记录的f00*.backblazeb2.com端点,找到对应的CNAME。实际上我(应该是)不能在Cloudflare上看见访问的具体URL和数据。

记录新的端点域名。

(可选)注册并添加域名

如果你想用自己的域名:

在https://dash.cloudflare.com/sign-up 注册。

官方教程:https://support.cloudflare.com/hc/zh-cn/articles/201720164-%E5%88%9B%E5%BB%BA-Cloudflare-%E5%B8%90%E6%88%B7%E5%B9%B6%E6%B7%BB%E5%8A%A0%E7%BD%91%E7%AB%99

设置CNAME

上文配置B2时记录了端点域名。

去Cloudflare,设置DNS,添加一个CNAME记录指向端点域名。一定要打开Cloudflare转发!

记录你设置的新域名,例如f002b2.cnbeining.com

Rclone

安装

Rclone的安装教程在https://rclone.org/install/ 。

最简单的安装方法是在Linux小鸡中运行curl https://rclone.org/install.sh | sudo bash

Windows小鸡:在https://rclone.org/downloads/ 找到安装包,下载,解压。下文中所有的rclone命令都用rclone.exe替换。

macOS:brew install rclone即可。

配置

在命令行中敲rclone config

e,回车。

输入后端昵称,例如b2。回车。

选择Backblaze B2:

找到Backblaze B2,输入数字。这里输入数字5。回车。

参考上面配置App Key的记录:account是上面的keyIDApplication KeyapplicationKey

hard_delete代表硬删除(彻底删除)或软删除(设成隐藏)。为了节省空间我选true

Edit advanced config? (y/n)这里按y进入高级设置。

一路回车使用默认值即可,除非你知道自己在做什么。

这里输入新端点域名:我这里输入f002b2.cnbeining.com

一路回车即可。

测试一下:

rclone ls b2:<你的仓库名>。你应该能看见仓库里的内容。

挂载

在小盘机上创建一个目录,然后把远程存储挂载过去,例如sudo mkdir /b2

开始挂载:rclone mount b2:/<你的仓库名> /b2 &

现在你可以把/b2目录当做本机使用。不建议大量读写(创建临时文件等),日常使用没有问题。

Hacking SearX Docker, Nginx and Cloudflare together the wrong way

SearX is a great meta search engine that aggrgate multiple engine’s result together, giving you privacy during searching.

A list of public instances can be found at https://searx.space/ – however it’s not possible to know what logging those public instances are putting up. Some public instances are using Cloudflare, which is OK – but some tends to set the senstivity too high which ruins the experience. Note Cloudflare can see everything – but for personal users you do need that to stop bots.

A better solution is to create your own instance, and share with your friends. The sharing step is as important as setting up – otherwise it’s effectly the same as you are using a single proxy. But think twice before setting up public instance unless you know what you are doing.

SearX has an official Docker Compose repo at https://github.com/searx/searx-docker – but I am already running Nginx on 443. So I need to hack the setup to make my current setup working with the new containers. Make sure you read https://github.com/searx/searx-docker#what-is-included- and understand which part is for what.

Grab this repo, edit .env file as instructed, and run ./start.sh once. Don’t worry about issues: we will hack them though.

Hacking Caddyfile

I should not use Caddy with Nginx but to make it working:

  1. Remove all morty related content
  2. If you want to use Cloudflare, hack Content-Security-Policy and add https://ajax.cloudflare.com/ in script-src 'self'; otherwise rocket loader won’t work.

Hacking searx/settings.yml

You need to change the Morty related stuffs at the end. Hardcode your Morty URL in, like https://search.fancy.tld/morty .

Hacking docker-compose.yml

  1. For Caddy, bind 80 to other ports. Like 4180:80.
  2. For morty: limit port 3000 to only localhost.
  3. For searx: hardcode morty related URL in.

Hacking .env

  1. Put localhost:4180 in host so Caddy won’t take port 80 from Nginx.
  2. Use HTTP only. We shall do SSL with Nginx.

Hacking rules.json

Remove the block deflate part if you need Cloudflare.

Hacking Nginx

Try this setup:

Note you must use upstream for reverse proxy or morty will complain.

With all the setup you should have something more or less usable. Wait for the checker to finish for optimized list of engines to enable – and note Qwant and DDG both uses Bing result, while Startpage is watered down Google.

If you want to set your SearX as default search engine for Chrome: visit your site, go to and your engine should be selectable. You may need to change the URL.

Customize Microsoft Sculpt Ergonomic Desktop on macOS

      6 Comments on Customize Microsoft Sculpt Ergonomic Desktop on macOS

Recently I got 2 sets of Microsoft Sculpt Ergonomic Desktop Keyboard & Mouse Bundle (https://www.microsoft.com/accessories/en-ca/products/keyboards/sculpt-ergonomic-desktop/l5v-00002) for use in office and at home.

Quick review:

  • The wrist rest for the keyboard is soft: but easy to get dirty
  • Note F keys are button rather than ordinary keys. Do not purchase if you need F keys often.
  • Key is easy to type on
  • Buttons on the mouse are soft
  • Note the keypad is separate. If you need the keypad often, consider https://www.microsoft.com/en-ca/p/surface-ergonomic-keyboard/90pnc9ljwpx9?activetab=pivot%3aoverviewtab
  • Note if you purchase the mouse and the keyboard separately you will have 2 USB-A dongles: but the full set only requires 1 dongle if you get the set version. You cannot separate the set: and dongle is not reprogrammable.

Another version of this keyboard exists as https://www.microsoft.com/en-ca/p/surface-ergonomic-keyboard/90pnc9ljwpx9?activetab=pivot%3aoverviewtab

To get this keyboard & mouse working on macOS you will need the following list of software:

all of them are open source.

Steps:

  1. Change the switch on the right corner of the keyboard to Fn
  2. Go to Karaviner-Elements.app, select the keyboard,
    • switch the left command and option keys
    • map right_gui to mouse5(Mouse buttons-button5)
    • Remap F7~F9 to match the keyboard symbols.
  3. Open Mos.app. Adjust the scrolling as needed. Maybe in\verse the scrolling.
  4. Open SensibleSideButtons.app. Enable it.

Result:

  • F keys will map the media keys
  • Command and Option keys match Mac keyboards
  • Scrolling is smoothed
  • Back key on the mouse is back; Windows key is forward(at least in Chrome)

Missing:

  • Calculator key: it’s not showing up in keyboard events
  • Double-tap is missing since no mouse would support tapping except for Magic Mouse 2

V2Ray WebSocket+TLS+Web+Nginx+CDN

      3 Comments on V2Ray WebSocket+TLS+Web+Nginx+CDN

由于SS挂的厉害,网上的教程又语焉不详,这里记录一下。

原理:用Nginx(Caddy)解TLS,V2Ray处理里面的连接。

不建议初学者直接上手搞WebSocket+TLS+Web+Nginx+CDN:容易出错的地方太多。

请务必读完全文再操作,特别是注意部分。

Preparation

WebSocket

WebSocket可以直接连接,或者套CDN。

Server

Client

注意

  • 如果挂Cloudflare:
    • 如果自签证书(真的没必要),SSL必须设flexible:否则CF会报证书错误
    • 如果使用正常SSL证书,SSL必须设Full:否则Nginx有可能随便丢一个站点过去
    • 如果是新域名:等SSL生成后才能操作
    • Cloudflare的免费证书只支持一级subdomain的SSL(*.domain.tld):如果域名是二级以上,请加钱或重新弄SSL。
  • 利用curl进行debug。在任何情况下,错误码不应该是404.

HTTP2 with Caddy

Cloudflare的免费WebSocket优先级不高:HTTP2有可能看网页更快。当然了,这个方案没有什么CDN能支持。

Server

Client

Note

  • Nginx不能做HTTP 2转发:因为作者觉得没必要。只能用Caddy。
  • 如果要使用CDN:虽然很多CDN支持HTTP2(例如Cloudflare),但是我们需要的是回源走HTTP2。目前还没有找到这种东西。

小丸工具箱入门操作教程

      1 Comment on 小丸工具箱入门操作教程

本教程为小丸工具箱入门操作教程,仅对小丸工具箱的重要功能作详细解释。可能某些功能通过小丸的后续更新变得有些不同,所以仅供学习参考。

本次以236版来讲解

选项页1:视频


这里是压制视频的地方
通常来说,你把视频添加到“单视频压制操作范围”里面,在“通用参数”里调试你所要的参数,点击压制即可。
批量压制同理,不过它点的是右下角的压制。
普通的压制只需要调一调CRF的数值和视频的分辨率即可,不需要搞其他更深奥的东西。

需要注意的是,小丸会自动识别跟视频在同一个文件夹的同名字幕文件,在“单视频压制操作范围”增加视频后,它会自动添加同文件名的字幕文件上去。
而在批量压制的框框里,视频文件名会变成蓝色,跟上图的效果一样,再勾上右边的“内嵌字幕”,就可以压进去了。如果字的颜色是黑的,那么就说明没关联到字幕。

视频选项详细解说

编码器

无论是32位还是64位,都无所谓,跟着系统走就行了。(压制4K或以上最好用64位编码器,否则容易出现爆内存)
而“X264_GCC”,是用GCC编译的X264版本。
至于怎么选择都没必要纠结,一般压制是没区别的。

音频模式:

小丸这里分为3个模式:复制、压制、无音频流。
复制模式:顾名思义就是直接复制过来,使用这个模式之前,你需要考虑原视频的音频是否能封装进你所压制好的容器里。
压制模式:该选项的设置在“音频”选项页中,设置是通用的。
无音频流 :那就是没有音频了。

分离器:

这里一般选项为auto,没啥事不要动它。
压制视频出来画音不同步,有很大一部分都是分离器出问题,要是出现这个问题,请提供压制日志到论坛或交流群内报错。

起始帧和编码帧数:

该选项默认为0,如果你只需要压制该视频的其中一小段,那么你只要在“起始帧”设置从第几帧开始压制,在“编码帧数”上设置要压多少帧即可。
需要注意的是,使用该功能一定要“无音频”压制

压制模式

CRF:

该参数的质量范围一般为1-51,一般设置21-25之间就可以,此值越大码率范围越低。21可以压制出高码率。网络播放则设为24即可。
质量是x264参数中的 CRF(Constant Rate Factor),这种码率控制方式是非常优秀的,以至于可以无需2pass压制,即使1pass也能实现非常好的码率分配利用。很多人在压片的时候不清楚应该给视频压到多少码率才比较好。CRF就是按需要来分配码率的。

2Pass:

一般是给控制体积和码率用的,压出来的画质好与坏,得看你定下了多少码率和你的片需要多少码率。
2pass正是有“压制需要两遍,浪费时间”、“出问题的概率多”、“压制出来的效果不太好”等缺点,我们一般都不推荐大家使用。

自定义:

就是给你写自己的参数用的,会直接覆盖掉小丸所有内置的X264参数。但不包括编码器、音频模式、起始帧和编码帧数这4个选项。
使用之前,请确定你对X264的参数知识有一定了解,不要无脑复制别人的压制参数就直接压。

选项页2:音频


无论是音频还是视频,都可以从这里处理成一个单独的音频文件
只要选择不同的编码器即可。
需要注意的是,这里的设置跟视频页的“压制音频”通用,单独压其他音频后,记得改回AAC编码器,否则会造成视频压制失败。
音频合并功能有点小问题,凑合能用,但不推荐用。

选项页3:常用


此选项页有3大功能
1. 一图流
2. 视频无损截取
3. 视频方向旋转

一图流

音频码率

压制音频到所填写的码率,如果不需要压制请勾选复制音频即可。
在勾选了“复制音频流”的情况下,请不要直接使用AAC格式音频,得把它封装成M4A或MP4格式才能正常运行。

FPS

最好填上23或者30比较常用的帧数,如果直接用1的话,调戏进度条会比较困难。

CRF

跟压制的CRF同理,默认就好。

时间

把音频拖进去后,小丸会自动识别该音频有多少秒,如识别有误,可手动更改。

其他

起始时刻和结束时刻

起始时刻和结束时刻:时间格式为 时:分:秒,设定时间只需要结束时刻大于起始时刻点击“截取”即可。
截取之后总时间可能会有点误差,属于正常现象,不影响使用。
根据ffmpeg的无损截取原理,是无法做到准确到秒来截取的,只能准确到该时间的关键帧来截取。
需要准确到秒,请使用PR或爱剪辑等编辑软件或者通过ffmpeg重编码的方式来准确截取。

Transpose

你要把画面怎样旋转,就选哪个,选好点击“旋转”即可。
该设置也得重编码。

选项卡4:封装

合并成MP4

可以把单独的H264和AAC文件封装到一起

FPS和PAR

万年不用该的选项,可以不理。

替换音频

可以把视频里的音频替换成添加进来音频

合并成MKV

这里仅提供一些基础的MKV封装功能,如需要用其他复杂功能,建议你使用MKVExtractGUI等软件

批量封装

视频只要是AVC+AAC格式的,都可在这里批量封装成MP4/MKV/FLV/AVI等格式。
需要注意的是,音频不是AAC格式的都会被转成AAC格式。

选项卡5: AVS

我就不多介绍了,会用的基本都懂,不懂的教起来也复杂,还是找个AVS教程看看吧。

选项卡6:MediaInfo

用来查看视频信息,报错必备。

选项卡7:设置

界面语言

支持简、繁、英、日 4种。

托盘模式

该功能仅仅是压制时托盘有个小丸图标,在图标中悬停会有简要的压制进度信息,并且压制完成后会弹出提示。

X264优先级

这个选项可以设置程序的优先级,不会大幅度提升或降低压制速度,顶多是能让你一边凑合着玩游戏一边压制。

X264线程

一般来说小丸的X264只能利用到16线程,所以让它auto就可以。如果需要限制线程,那就根据自己最大的框框数量,减去需要保留的框框数量,得出来的和就是X264的线程。

X264自定义命令行

这个自定义,它不会把小丸的界面参数全部覆盖掉,只会覆盖掉内置参数。
即保留--crf 24.0 --threads 16这两项参数。

预览播放器

可以在这里指定AVS的预览播放器。

退出程序时删除所有临时文件

该功能开启后,在退出软件时,../MarukoToolbox/tools文件夹里的批处理等临时文件会自动删除。

启动X265

勾上了就可以用X265了,该功能还在测试中,所以默认不开启。

还原默认设置

小丸被玩坏了可以点这里。゚∀゚)σ

查看日志

点击即可查看最新的日志文件,如果要查找以往的日志文件,可以去 ../ MarukoToolbox/logs 文件夹里查找。

删除日志

点击后整个logs文件夹被删除。

From anywhere to AWS Lambda in one line with Zappa

      No Comments on From anywhere to AWS Lambda in one line with Zappa

The problem

We always want to do continus integration and deployment with our repo. Bitbucket comes with handy build function.

Version releasing with Zappa is easy: zappa update xxx will make a release, and zappa rollback xxx -n 3 would revert the changes.

But Zappa is currently broken on Python 3.7 as Zappa is using async as package name, while Python 3.7 shall use async and await as reserved names.

Locally I use Python 3.7 with macOS, but I have to support Windows + macOS + Ubuntu + CentOS: how can I quickly make release everywhere?

Solution

Local

Refer to https://blog.zappa.io/posts/docker-zappa-and-python3.

LambCI has made a couple of Docker images that would simulate AWS Lambda, located at https://github.com/lambci/docker-lambda , which provides handy shell access.

With CI

With some hacking we can make a Docker image for release, as in https://blog.zappa.io/posts/simplified-aws-lambda-deployments-with-docker-and-zappa . But this image only supports Python 2.7.

A Python 3.6 version is located at https://cloud.docker.com/repository/docker/cnbeining/zappa3 . And we can have a one-liner:

docker run -e AWS_SECRET_ACCESS_KEY=xxxxxxxxx -e AWS_ACCESS_KEY_ID=AKXXXXXXXXXXX -e AWS_DEFAULT_REGION=us-west-2 -v $(pwd):/var/task --rm cnbeining/zappa3 bash -c "virtualenv -p python3 docker_env && source docker_env/bin/activate && pip install -r requirements.txt && zappa update && rm -rf docker_env"

This command will create a environment, attach your current folder, install all the requirements, update the version, and remove all the garbage.

One note: DO NOT SET profile in zappa_settings.json. This image will automatically login with your key.

Reference:

https://blog.zappa.io/posts/continuous-zappa-deployments-with-travis

Flask from Docker to Lambda with Zappa: the more-or-less complete guide

TLDR

Step-by-step guide of how FleetOps migrate the Docker-based Flask API to AWS Lambda.

History

At FleetOps.ai we use Docker extensively when building APIs. Our API is built on Flask, with microservices supporting async features.

Since we are moving microservices to AWS Lambda … What if the main API could also run on Lambda?

AWS Lambda & Serverless

Serverless is probably the hottest word in the DevOps world in 2018. Does not sound very interesting?

Compared to SaaS(Google App Engine, Heroku, Openshift V2, Sina App Engine, etc.): serverless does not have severe vendor lock-in problem. Most of the time you do not need to edit ANYTHING to migrate to serverless. You CAN choose to write the code in a SaaS way: and if you don’t fancy that a DIY approach is still available. In this case I did not make any change to the original codebase!

Compared to Docker: although Docker is more flexible and you have access to a full Linux OS within the VM, it’s still hard to manage when scaling. Kubernetes is good: but the burden for DevOps is dramatic. At FleetOps we do not want to put so much energy into DevOps: not to say hobby project.

Compared to Web Hosting: serverless supports more languages(Java, Node, etc.) which are not possible to get in the Hosting world.

Problem/limits with AWS Lambda

To name a few:

  • Does not support ALL the languages like Docker, and definitely not ALL the versions of Python. AWS is working on super lightweight OS image so maybe we can see something different?
  • Have to bring your binary/library should you want to use any special software, and they have to be statically linked, while with Docker you can do anything you want. Well, does not sound very bad, but:
  • The size limit of code: if you love 3rd party library it may be very hard to put everything into one zipball. Well technically you can grab them on the fly upon function invoked from S3, BUT:
  • Cold start problem: you have absolutely no control the life cycle of those function. God bless you if your function needs 10s to start.
  • Hard max runtime: 900s is the limit. Maybe you can get it raised but YMMV.
  • Stateless: Like container committing suicide after every invoke.
  • No access to special hardware, like GPU.
  • No debugger: do some print()s instead.
  • Confusing networking: I will try to sort out this issue in this article.

So if your task is:

  • not require any special technology, and uses the most common stack
  • stateless, or is able to recover state from other services(which should be the standard for every API – at least in FleetOps we ensure that every API call shall be stateless)
  • one task does not run forever and does not consume lots of memory
  • not really benefiting from JIT or similar caching
  • not super huge
  • not using fancy hardware
  • having an uneven workload

Then you could benefit from AWS Lambda.

The Guide

1. Get ready

We use Python 3.6 for the API for now.

Get a requirement.txt ready. Not there yet? pip freeze > requirements.txt.

On your dev machine, make a virtual environment: (ref: https://docs.python-guide.org/dev/virtualenvs/)

Install Zappa(https://github.com/Miserlou/Zappa ): pip install zappa

Get your AWS CLI ready: pip install boto3 and refer to steps in https://pypi.org/project/boto3/ . Make sure that account has full access to S3, Lambda, SQS, API Gateway, and the whole network stack.

2. Some observations and calculations:

  • Where is your main function? Make a note of that.
  • How much memory do you need? If you cannot provide a definite number yet, let it here.
  • What is your target VPC & security group? Note their IDs.
  • What 3rd party binary do you need? Compile them with statically linked library – you cannot easily call apt-get on the remote machine!
  • Do you need any environment variables? There are different ways of setting them, and I am using the easiest approach – putting them in the config JSON.

3. Get the Internet right!

Further reading: https://gist.github.com/reggi/dc5f2620b7b4f515e68e46255ac042a7

Quote from @reggi ‘s article:

So it might be really unintuitive at first but lambda functions have three states.
1. No VPC, where it can talk openly to the web, but can’t talk to any of your AWS services.
2. VPC, the default setting where the lambda function can talk to your AWS services but can’t talk to the web.
3. VPC with NAT, The best of both worlds, AWS services and web.

Use 1. if you do not need this function to access any AWS service, or you only need the function to access them via the Internet. Use 2. if you are building a private API. And for FleetOps, we are going down path 3.

Note that not all the AWS services are accessible by VPC: e.g., S3 and RDS are accessible by VPC, while SQS and DynamoDB would require Internet access, even you are calling from within Lambda.

My recommended step is:

  1. Create Internet Gateway.

  1. Create 4 subnets.

  1. Create NAT Gateway.

  1. Create Route table.

Take note of the 3 private-faced subnet ids.

We will use Zappa to configure the networking. Note if you want to deploy the function to multiple AZ, you may need to do the steps multiple times, once at each AZ.

4. Wrap it up

Get back to your virtual env, and active it.

Do a zappa init. You will be asked the following questions:

Use whatever name: and you can carry on the stage’s configuration for further stages.

By default, Zappa will only use this bucket when uploading/updating the function.

Put in the entrance function.

Depends on your use case.

Now you may want to edit the zappa_settings.json: all the arguments are at https://github.com/Miserlou/Zappa#advanced-settings but this is the basic one that get our API running:

There are TONS of settings Zappa provides but I am not using all them: You can use a selective set of feature to make sure you do not have vendor lock-in. For example, Lambda can handle URL routing by itself but I am not using it to avoid any kind of lock-in. By doing so you can easily take the code and put them back on the container if you wish.

Zappa does provide some exciting feature:

  • Setting AWS Environment variables: If you prefer to put the secret key in another place
  • Auto packing huge project: if your project is >50M, Zappa will handle that.
  • Keep warm: Use CloudWatch to make sure there is one function running.

Save zappa_settings.json.

5. PROFIT!

Do a pip install -r requirement.txt to install all the packages.

Now do a zappa deploy.

You would see:

And you now have a serverless API ready to serve!

6. Clean up

You want to do the following tasks to save $$$, boost performance and secure the setup:

  • View some CloudWatch log and set the memory to a reasonable value afterwards.
  • Adjust warmer period.
  • Adjust API Gateway caching.
  • Setup cronjobs if you have them: either with Zappa or with CloudWatch.
  • Change the scope of IAM user for Zappa: the default one is super powerful.
  • Adjust X-Ray if you need it.

Conclusion

Migrating Flask API to serverless could be painless. I did not adjust one single line of code: and there is no vendor lock-in as every step can be reproduced by Dockerfile.

Good luck with your journey with serverless!

备忘:SOCKS和SOCKS/HTTP代理的连接

      1 Comment on 备忘:SOCKS和SOCKS/HTTP代理的连接

Use case:

  • 直接SOCKS肯定过不去GFW
  • 公有的SS看Google会跳验证码或者IP黑掉
  • Chrome的SOCKS代理不支持密码验证

方法:

  1. 本地安装Proxifier。配置好SS 不要使用全局模式,只监听某个端口。准备好可以使用的SOCKS/HTTP代理。
  2. 在Proxifier中分别配置好两个代理。
  3. 如图设置:

  1. 在Proxifier中,对浏览器使用这个chain,如图所示(例子是curl):

  1. 现在你的浏览器走SS翻墙 但是IP换了。

$ curl "http://ip-api.com/json"
{"as":"AS32489 Amanah Tech Inc.","city":"Toronto","country":"Canada","countryCode":"CA","isp":"Amanah Tech","lat":43.6683,"lon":-79.4205,"org":"Amanah Tech","query":"184.75..xxx","region":"ON","regionName":"Ontario","status":"success","timezone":"America/Toronto","zip":"M6G"}