Using Docker and Docker-Compose on macOS with Multipass

      No Comments on Using Docker and Docker-Compose on macOS with Multipass

In the light of recent news that
Docker Desktop for Windows and macOS is no longer free, efforts have been put in to search for a replacement.

Docker Desktop?

Docker Desktop used to be Docker Toolbox, which is based on VirtualBox. Docker Desktop is able to utilize Hyperkit on macOS which makes the performance much better compared to Type II hypervisor.

In my mind a good in-place replacement should be:

  • Runs single Dockerfile
  • Supports volume mounting and port forwarding
  • Runs docker-compose.yml
  • Easy to setup
  • Supports Windows and macOS(Docker Engine on Linux remains open-sourced)
  • Works with built-in debugger for common IDEs(IntelliJ & VSCode)
  • Works with IDE’s built-in support that’s based on Docker socket
  • Supports ARM Mac

Common replacements proposed includes:

  • Podman: a Redhat-backed solution that runs on OCI. Spins up a Podman-flavored VM. Very limited support for Docker-Compose. Probably requires reworking Dockerfile. Volume mounting is painful. Cannot be used with IDE’s built-in support.
  • Microk8s & k3s: Kubenetes. Won’t work with Docker-Compose.


This is where Multipass comes to shine:

  • Utilized Hyperkit, Hyper-V, and Virtualbox for best performance
  • Handles volume mounting nicely
  • Easy to setup
  • Networking is easy
  • Volume mounting is simple – at least when using Hyperkit
  • Native ARM support

Downsides include:

  • Ubuntu. Not even Debian. Bad news for CentOS lovers.
  • Overlapping function with Vagrant. Can’t deny that.

The actual setup

Steps based on macOS Catalina, 10.15.7.

Install Multipass

Download installer
here. You DO NOT need to set driver as VirtualBox – leave it as is and Hyperkit will be used as default!

You can try running brew install multipass.

Make sure that you give multipassd Full Disk Access – otherwise mounting will fail!

Create and setup your instance

I am calling my instance as

Note the default disk size is merely 5GiB – not enough for setting up Docker. To give it a bit more space:

multipass launch fakedocker -d 20G

Use -c to add more CPU cores, -m to add more RAM. Documentation

One-liner with cloud-init

You may want to change Docker-Compose’s version to the latest.

Save this file as cloud-init.yaml, and run:

multipass launch fakedocker -d 20G --cloud-init cloud-init.yaml

If having any problem, run multipass shell fakedocker to SSH into the VM. Logs are located at /var/log/cloud-init.log.

Manually set up your instance

Create VM

Use multipass launch fakedocker -d 20G to create the instance.

Run multipass shell fakedocker to get into the instance.

Setup Docker and other stuff

Follow this guide to install Docker:

Also this post-install guide to setup groups:

For installing Docker-Compose:

Run sudo apt-get install -y avahi-daemon libnss-mdns to install Avahi to enable Bonjour so you can access the box by fakedocer.local. This will comes in handy in the following setup.

Edit Docker’s Systemd config to expose both port and Unix socket: TCP is for remote debugger on your local machine, the socket is for running Docker command locally:

Create a folder at /lib/systemd/system/docker.service.d/ add a file overwrite.conf with the following content:


to make sure that docker command works on the VM.

Mounting local folder

Run multipass mount ~ fakedocker to attach your home directory to the new VM. This will make sure that you don’t need to calculate where you are mounting.

Quick sanity test

On fakedocker VM,

  • Run docker pscommand: Docker should be able to run. You may need to log out and re-login for group change to take effect.
  • Run nc -zv 2375to make sure that Docker Engine is taking traffic from TCP.
  • Run ls /User/<your_username>to make sure that volume attaching is successful: use catto read a file, and write something random to make sure that the attached volume is readable and writable.

On your local machine:

  • do a nc -zv fakedocker.local 2375to make sure that your local debugger can communicate with the Docker instance on fakedocker.

Now you’ve got a nice environment where you can use
as if you are on your local machine.

Setting up Docker on Mac

Run export DOCKER_HOST="tcp://fakedocker.local:2375" or put it in your ~/.zshrc to make the VM your default Docker host.

Follow to setup docker command locally, and to setup docker-compose.

Setting up remote debugger

I am using PyCharm with Docker-Compose with attached volume in docker-compose.yml as an example:

Create a new Docker machine

PyCharm should be able to connect to this instance.

Setup a new Interpreter

Make sure that you do Path mappings as shown: otherwise, debugger won’t be able to run.

Setup debug config

Listen to – otherwise, you won’t be able to visit the web service.

That should be it! Trigger debugging – if you are listening to port 8000, go to http://fakedocker.local:8000 to visit your site. Setup port forwarding in docker-compose.yml if you want to expose more services, like your database.

Common problems

Multipass VM stuck at “Starting”/ Multipass is stuck on any command

sudo pkill multipassd
and redo your last step. multipassd is a daemon that checks VM: killing this process does not change the state of VM. (So what’s the point of making it a daemon?)

Multipass VM is botched

Run multipass delete <name_of_vm> && multipass purge to completely remove the VM. 

Multipass VM cannot read mounted folder on computer reboot

Reattach the volume:
multipass unmount ~ fakedocker && multipass mount ~ fakedocker.

Cannot connect to remote Docker

Make sure that you’ve edited Docker’s startup command to include TCP listening on

Need to debug issues when using cloud-init

Logs are located at /var/log/cloud-init.log.

docker command on VM won’t run

  • Make sure that Docker’s startup command also listens on Unix socket.
  • That socket file needs to be 666.

IDE cannot connect to debugger: cannot find debugger file(especially Pycharm)

Mount your project home to


使用Backblaze B2+Cloudflare+rclone为小盘机(免费)增加空间

小盘机的空间经常不够用:我们可以把Backblaze B210G免费空间搬到小鸡上。


  • 接码手机号和邮箱
  • (可选)一个顶级域名,供注册Cloudflare



Rclone最大的作用是同步文件。Rclone支持很多稀奇古怪的数据源,以及很多稀奇古怪的功能 – 其中包括将远程文件挂载成FUSE文件系统。

利用类似本文的方法,也可以将Google Drive,Onedrive,Dropbox,远程大盘鸡等挂载成文件系统。

Backblaze B2

Backblaze的服务器在美国西部,虽然只有一个机房但是便宜:每T存储每月5.12 USD。


  1. Backblaze加入了Cloudflare的Bandwidth Alliance,下载流量免费;上传不收费。

  2. Backblaze B2是企业级存储:8个9的可靠性,3个9的可用性。不会像个人网盘删文件或限制API请求频率。

  3. 每个账户送10G存储空间,每日1G下载流量,2500次下载请求。对于白嫖够用了。


Backblaze B2

注册Backblaze B2

点击 注册。



在 处,点击“Create a Bucket”。

  • Bucket Unique Name: 仓库的名字。必须全局唯一,仅限字母和数字。
  • Files in Bucket are: 公开还是私密仓库。私密(private)即可,除非你不在乎公开文件。
  • Object Lock: 锁定文件一段时间内不能删除。不需要开启。

记录仓库的名字。点击蓝色的Create a Bucket 创建仓库。

和所有的对象存储一样,B2默认保存文件的所有版本:如果要修改,点击仓库的Lifecycle Settings

  • Keep all versions of the file (default):默认保存所有版本
  • Keep only the last version of the file:只保存最后一个版本。我会使用这个节省空间。
  • Keep prior versions for this number of days:在X天内保存旧版本。
  • Use custom lifecycle rules:按文件前缀自定义多少天隐藏,多少天删除文件。

创建APP Key

默认的key不能使用:在 创建一个新key。

点击Add a New Application Key

  • Name of Key: 名字,供你自己参考
  • Allow access to Bucket(s): 你可以让key只能访问某个仓库。
  • Type of Access: 只读,只写或读写都可以。除非你知道你在做什么,否则选择Read and Write读写均可。
  • File name prefix: 只能访问前缀为此项的文件。除非你知道你在做什么,否则留空。
  • Duration (seconds): 有效期。除非你知道你在做什么,否则留空。

点击Create New Key。你会看见创建的key:立即保存,这些信息只出现一次。



首先找到你的数据端点:在 ,点击你创建的仓库;

点击Upload,上传一个文件 – 哪怕只有1字节也行。上传成功后,点击这个文件。

你会看见很多链接:找到Friendly URL,看域名是什么。例如代表



我用自己的域名(2012年至今,应该永久续费)和CF Pro做了如下设置:


如果你想用自己的域名,按下面的流程注册Cloudflare并绑定域名,设置CNAME:如果懒得搞,用我提供的域名即可 – 从上文记录的f00*.backblazeb2.com端点,找到对应的CNAME。实际上我(应该是)不能在Cloudflare上看见访问的具体URL和数据。




在 注册。








Rclone的安装教程在 。

最简单的安装方法是在Linux小鸡中运行curl | sudo bash

Windows小鸡:在 找到安装包,下载,解压。下文中所有的rclone命令都用rclone.exe替换。

macOS:brew install rclone即可。


在命令行中敲rclone config



选择Backblaze B2:

找到Backblaze B2,输入数字。这里输入数字5。回车。

参考上面配置App Key的记录:account是上面的keyIDApplication KeyapplicationKey


Edit advanced config? (y/n)这里按y进入高级设置。





rclone ls b2:<你的仓库名>。你应该能看见仓库里的内容。


在小盘机上创建一个目录,然后把远程存储挂载过去,例如sudo mkdir /b2

开始挂载:rclone mount b2:/<你的仓库名> /b2 &


Hacking SearX Docker, Nginx and Cloudflare together the wrong way

SearX is a great meta search engine that aggrgate multiple engine’s result together, giving you privacy during searching.

A list of public instances can be found at – however it’s not possible to know what logging those public instances are putting up. Some public instances are using Cloudflare, which is OK – but some tends to set the senstivity too high which ruins the experience. Note Cloudflare can see everything – but for personal users you do need that to stop bots.

A better solution is to create your own instance, and share with your friends. The sharing step is as important as setting up – otherwise it’s effectly the same as you are using a single proxy. But think twice before setting up public instance unless you know what you are doing.

SearX has an official Docker Compose repo at – but I am already running Nginx on 443. So I need to hack the setup to make my current setup working with the new containers. Make sure you read and understand which part is for what.

Grab this repo, edit .env file as instructed, and run ./ once. Don’t worry about issues: we will hack them though.

Hacking Caddyfile

I should not use Caddy with Nginx but to make it working:

  1. Remove all morty related content
  2. If you want to use Cloudflare, hack Content-Security-Policy and add in script-src 'self'; otherwise rocket loader won’t work.

Hacking searx/settings.yml

You need to change the Morty related stuffs at the end. Hardcode your Morty URL in, like https://search.fancy.tld/morty .

Hacking docker-compose.yml

  1. For Caddy, bind 80 to other ports. Like 4180:80.
  2. For morty: limit port 3000 to only localhost.
  3. For searx: hardcode morty related URL in.

Hacking .env

  1. Put localhost:4180 in host so Caddy won’t take port 80 from Nginx.
  2. Use HTTP only. We shall do SSL with Nginx.

Hacking rules.json

Remove the block deflate part if you need Cloudflare.

Hacking Nginx

Try this setup:

Note you must use upstream for reverse proxy or morty will complain.

With all the setup you should have something more or less usable. Wait for the checker to finish for optimized list of engines to enable – and note Qwant and DDG both uses Bing result, while Startpage is watered down Google.

If you want to set your SearX as default search engine for Chrome: visit your site, go to and your engine should be selectable. You may need to change the URL.

Customize Microsoft Sculpt Ergonomic Desktop on macOS

      7 Comments on Customize Microsoft Sculpt Ergonomic Desktop on macOS

Recently I got 2 sets of Microsoft Sculpt Ergonomic Desktop Keyboard & Mouse Bundle ( for use in office and at home.

Quick review:

  • The wrist rest for the keyboard is soft: but easy to get dirty
  • Note F keys are button rather than ordinary keys. Do not purchase if you need F keys often.
  • Key is easy to type on
  • Buttons on the mouse are soft
  • Note the keypad is separate. If you need the keypad often, consider
  • Note if you purchase the mouse and the keyboard separately you will have 2 USB-A dongles: but the full set only requires 1 dongle if you get the set version. You cannot separate the set: and dongle is not reprogrammable.

Another version of this keyboard exists as

To get this keyboard & mouse working on macOS you will need the following list of software:

all of them are open source.


  1. Change the switch on the right corner of the keyboard to Fn
  2. Go to, select the keyboard,
    • switch the left command and option keys
    • map right_gui to mouse5(Mouse buttons-button5)
    • Remap F7~F9 to match the keyboard symbols.
  3. Open Adjust the scrolling as needed. Maybe in\verse the scrolling.
  4. Open Enable it.


  • F keys will map the media keys
  • Command and Option keys match Mac keyboards
  • Scrolling is smoothed
  • Back key on the mouse is back; Windows key is forward(at least in Chrome)


  • Calculator key: it’s not showing up in keyboard events
  • Double-tap is missing since no mouse would support tapping except for Magic Mouse 2

V2Ray WebSocket+TLS+Web+Nginx+CDN

      3 Comments on V2Ray WebSocket+TLS+Web+Nginx+CDN











  • 如果挂Cloudflare:
    • 如果自签证书(真的没必要),SSL必须设flexible:否则CF会报证书错误
    • 如果使用正常SSL证书,SSL必须设Full:否则Nginx有可能随便丢一个站点过去
    • 如果是新域名:等SSL生成后才能操作
    • Cloudflare的免费证书只支持一级subdomain的SSL(*.domain.tld):如果域名是二级以上,请加钱或重新弄SSL。
  • 利用curl进行debug。在任何情况下,错误码不应该是404.

HTTP2 with Caddy





  • Nginx不能做HTTP 2转发:因为作者觉得没必要。只能用Caddy。
  • 如果要使用CDN:虽然很多CDN支持HTTP2(例如Cloudflare),但是我们需要的是回源走HTTP2。目前还没有找到这种东西。


      1 Comment on 小丸工具箱入门操作教程










无音频流 :那就是没有音频了。







质量是x264参数中的 CRF(Constant Rate Factor),这种码率控制方式是非常优秀的,以至于可以无需2pass压制,即使1pass也能实现非常好的码率分配利用。很多人在压片的时候不清楚应该给视频压到多少码率才比较好。CRF就是按需要来分配码率的。








1. 一图流
2. 视频无损截取
3. 视频方向旋转












起始时刻和结束时刻:时间格式为 时:分:秒,设定时间只需要结束时刻大于起始时刻点击“截取”即可。














选项卡5: AVS






支持简、繁、英、日 4种。








即保留--crf 24.0 --threads 16这两项参数。










点击即可查看最新的日志文件,如果要查找以往的日志文件,可以去 ../ MarukoToolbox/logs 文件夹里查找。



From anywhere to AWS Lambda in one line with Zappa

      No Comments on From anywhere to AWS Lambda in one line with Zappa

The problem

We always want to do continus integration and deployment with our repo. Bitbucket comes with handy build function.

Version releasing with Zappa is easy: zappa update xxx will make a release, and zappa rollback xxx -n 3 would revert the changes.

But Zappa is currently broken on Python 3.7 as Zappa is using async as package name, while Python 3.7 shall use async and await as reserved names.

Locally I use Python 3.7 with macOS, but I have to support Windows + macOS + Ubuntu + CentOS: how can I quickly make release everywhere?



Refer to

LambCI has made a couple of Docker images that would simulate AWS Lambda, located at , which provides handy shell access.

With CI

With some hacking we can make a Docker image for release, as in . But this image only supports Python 2.7.

A Python 3.6 version is located at . And we can have a one-liner:

docker run -e AWS_SECRET_ACCESS_KEY=xxxxxxxxx -e AWS_ACCESS_KEY_ID=AKXXXXXXXXXXX -e AWS_DEFAULT_REGION=us-west-2 -v $(pwd):/var/task --rm cnbeining/zappa3 bash -c "virtualenv -p python3 docker_env && source docker_env/bin/activate && pip install -r requirements.txt && zappa update && rm -rf docker_env"

This command will create a environment, attach your current folder, install all the requirements, update the version, and remove all the garbage.

One note: DO NOT SET profile in zappa_settings.json. This image will automatically login with your key.


Flask from Docker to Lambda with Zappa: the more-or-less complete guide


Step-by-step guide of how FleetOps migrate the Docker-based Flask API to AWS Lambda.


At we use Docker extensively when building APIs. Our API is built on Flask, with microservices supporting async features.

Since we are moving microservices to AWS Lambda … What if the main API could also run on Lambda?

AWS Lambda & Serverless

Serverless is probably the hottest word in the DevOps world in 2018. Does not sound very interesting?

Compared to SaaS(Google App Engine, Heroku, Openshift V2, Sina App Engine, etc.): serverless does not have severe vendor lock-in problem. Most of the time you do not need to edit ANYTHING to migrate to serverless. You CAN choose to write the code in a SaaS way: and if you don’t fancy that a DIY approach is still available. In this case I did not make any change to the original codebase!

Compared to Docker: although Docker is more flexible and you have access to a full Linux OS within the VM, it’s still hard to manage when scaling. Kubernetes is good: but the burden for DevOps is dramatic. At FleetOps we do not want to put so much energy into DevOps: not to say hobby project.

Compared to Web Hosting: serverless supports more languages(Java, Node, etc.) which are not possible to get in the Hosting world.

Problem/limits with AWS Lambda

To name a few:

  • Does not support ALL the languages like Docker, and definitely not ALL the versions of Python. AWS is working on super lightweight OS image so maybe we can see something different?
  • Have to bring your binary/library should you want to use any special software, and they have to be statically linked, while with Docker you can do anything you want. Well, does not sound very bad, but:
  • The size limit of code: if you love 3rd party library it may be very hard to put everything into one zipball. Well technically you can grab them on the fly upon function invoked from S3, BUT:
  • Cold start problem: you have absolutely no control the life cycle of those function. God bless you if your function needs 10s to start.
  • Hard max runtime: 900s is the limit. Maybe you can get it raised but YMMV.
  • Stateless: Like container committing suicide after every invoke.
  • No access to special hardware, like GPU.
  • No debugger: do some print()s instead.
  • Confusing networking: I will try to sort out this issue in this article.

So if your task is:

  • not require any special technology, and uses the most common stack
  • stateless, or is able to recover state from other services(which should be the standard for every API – at least in FleetOps we ensure that every API call shall be stateless)
  • one task does not run forever and does not consume lots of memory
  • not really benefiting from JIT or similar caching
  • not super huge
  • not using fancy hardware
  • having an uneven workload

Then you could benefit from AWS Lambda.

The Guide

1. Get ready

We use Python 3.6 for the API for now.

Get a requirement.txt ready. Not there yet? pip freeze > requirements.txt.

On your dev machine, make a virtual environment: (ref:

Install Zappa( ): pip install zappa

Get your AWS CLI ready: pip install boto3 and refer to steps in . Make sure that account has full access to S3, Lambda, SQS, API Gateway, and the whole network stack.

2. Some observations and calculations:

  • Where is your main function? Make a note of that.
  • How much memory do you need? If you cannot provide a definite number yet, let it here.
  • What is your target VPC & security group? Note their IDs.
  • What 3rd party binary do you need? Compile them with statically linked library – you cannot easily call apt-get on the remote machine!
  • Do you need any environment variables? There are different ways of setting them, and I am using the easiest approach – putting them in the config JSON.

3. Get the Internet right!

Further reading:

Quote from @reggi ‘s article:

So it might be really unintuitive at first but lambda functions have three states.
1. No VPC, where it can talk openly to the web, but can’t talk to any of your AWS services.
2. VPC, the default setting where the lambda function can talk to your AWS services but can’t talk to the web.
3. VPC with NAT, The best of both worlds, AWS services and web.

Use 1. if you do not need this function to access any AWS service, or you only need the function to access them via the Internet. Use 2. if you are building a private API. And for FleetOps, we are going down path 3.

Note that not all the AWS services are accessible by VPC: e.g., S3 and RDS are accessible by VPC, while SQS and DynamoDB would require Internet access, even you are calling from within Lambda.

My recommended step is:

  1. Create Internet Gateway.

  1. Create 4 subnets.

  1. Create NAT Gateway.

  1. Create Route table.

Take note of the 3 private-faced subnet ids.

We will use Zappa to configure the networking. Note if you want to deploy the function to multiple AZ, you may need to do the steps multiple times, once at each AZ.

4. Wrap it up

Get back to your virtual env, and active it.

Do a zappa init. You will be asked the following questions:

Use whatever name: and you can carry on the stage’s configuration for further stages.

By default, Zappa will only use this bucket when uploading/updating the function.

Put in the entrance function.

Depends on your use case.

Now you may want to edit the zappa_settings.json: all the arguments are at but this is the basic one that get our API running:

There are TONS of settings Zappa provides but I am not using all them: You can use a selective set of feature to make sure you do not have vendor lock-in. For example, Lambda can handle URL routing by itself but I am not using it to avoid any kind of lock-in. By doing so you can easily take the code and put them back on the container if you wish.

Zappa does provide some exciting feature:

  • Setting AWS Environment variables: If you prefer to put the secret key in another place
  • Auto packing huge project: if your project is >50M, Zappa will handle that.
  • Keep warm: Use CloudWatch to make sure there is one function running.

Save zappa_settings.json.


Do a pip install -r requirement.txt to install all the packages.

Now do a zappa deploy.

You would see:

And you now have a serverless API ready to serve!

6. Clean up

You want to do the following tasks to save $$$, boost performance and secure the setup:

  • View some CloudWatch log and set the memory to a reasonable value afterwards.
  • Adjust warmer period.
  • Adjust API Gateway caching.
  • Setup cronjobs if you have them: either with Zappa or with CloudWatch.
  • Change the scope of IAM user for Zappa: the default one is super powerful.
  • Adjust X-Ray if you need it.


Migrating Flask API to serverless could be painless. I did not adjust one single line of code: and there is no vendor lock-in as every step can be reproduced by Dockerfile.

Good luck with your journey with serverless!