Jekyll2021-06-15T08:52:34+00:00https://blog.fugoes.xyz/feed.xmlFugoes’s BlogTowards better Windows VM under GNU/Linux2020-02-14T12:27:00+00:002020-02-14T12:27:00+00:00https://blog.fugoes.xyz/2020/02/14/Towards-better-Windows-VM-under-GNU-Linux<p>Recently, I have to use some on-line meeting softwares whose desktop version support only Windows. So that once again, I need to install a Windows virtual machine on my laptop running Debian. I used to use qemu with kvm which deliver great IO performance with virtio drivers. However, its graphics performance is unbearable, even with the spice protocol and the qxl driver installed. I need to find a better Windows VM solution for GNU/Linux. Actually, there are lots of good VM softwares for GNU/Linux, for example, VirtualBox, qemu with kvm, and vmware series.</p>
<h1 id="qemu">qemu?</h1>
<p>I tried qemu with the VMWare SVGA-II compatible adapter as my first attempt, a.k.a the <code class="language-plaintext highlighter-rouge">-vga vmware</code> option. It actually performs better than the default standard VGA adapter, though still with minor screen tearing. At first, I think the default driver in the Windows guest is a bottleneck for achieving the full performance of this adapter, and installing the driver provided in vmware’s guest tools would help. So I managed to extract the vmware’s guest tools: I download the newest ISO for the tools from https://packages.vmware.com/tools/esx/latest/windows/x86/. According to some web pages, running the <code class="language-plaintext highlighter-rouge">setup.exe</code> in the ISO with a <code class="language-plaintext highlighter-rouge">/a</code> parameter would extract all the drivers included in the ISO. However, this binary refuses to execute under non-vmware guest environments. So I downloaded and installed vmware workstation player which is free of charge for non-commercial usage. Then install a Windows VM inside the player. And finally succeed extracting the video driver. After I installed this video driver inside the qemu’s Windows guest, the VM crashed with vmware related error message from qemu. At first glance, this seems like some version miss match between the driver and qemu. To verify it, I downloaded an older version of the tools whose release date matches my qemu version, but it still does not work, though with different error messages.</p>
<p>Then I tried qemu with the spice protocol. Spice is a remote graphics access protocol, but there are thousands of articles about its great performance on the Internet, so I think it deserves a try. But after appending <code class="language-plaintext highlighter-rouge">-vga qxl -spice port=5900,addr=127.0.0.1,disable-ticketing </code> options to qemu command line, and install spice drivers on the Windows guest, the GUI feels even worse than the default VGA adapter with more CPU consumption (for image encoding on the VM side and decoding on the remote side?). spice should be a great protocol for remote access over Internet, but it is not-so-optimized for local usage.</p>
<h1 id="virtualbox">VirtualBox?</h1>
<p>Then I gave VirtualBox a try.</p>
<p>For qemu, <code class="language-plaintext highlighter-rouge">virtio-scsi</code> disk driver support passing through the <code class="language-plaintext highlighter-rouge">unmap</code> command with the following options:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nt">-drive</span> <span class="k">if</span><span class="o">=</span>none,file<span class="o">=</span></path/to/raw/block/device>,format<span class="o">=</span>raw,id<span class="o">=</span>hd,discard<span class="o">=</span>unmap <span class="se">\</span>
<span class="nt">-device</span> virtio-scsi-pci,id<span class="o">=</span>scsi <span class="se">\</span>
<span class="nt">-device</span> scsi-hd,drive<span class="o">=</span>hd <span class="se">\</span>
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">unmap</code> command is used by many thin provision block device (e.g. zfs’s zvol, lvm’s thinpool) to reclaim space: when the guest VM issues trim on its filesystem (<code class="language-plaintext highlighter-rouge">fstrim</code> under GNU/Linux), the filesystem’s driver would issue <code class="language-plaintext highlighter-rouge">unmap</code> command on those unused space for the block device. Unlike normal trim on SSD drivers, when these thin provision block device receives <code class="language-plaintext highlighter-rouge">unmap</code> command, it would release the space used by these blocks. This feature is extremely useful for Windows VM. For example, on each major version update, a <code class="language-plaintext highlighter-rouge">windows.old</code> folder would be create. Without the <code class="language-plaintext highlighter-rouge">unmap</code> passing through, deleting <code class="language-plaintext highlighter-rouge">windows.old</code> would not release the space on the host side.</p>
<p>After some searching, I found something in VirtualBox’s document: https://www.virtualbox.org/manual/ch08.html#vboxmanage-storageattach. According to the document, though not supported in GUI, the user could use CLI to enable this feature:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>vboxmanage storageattach <VM name> <span class="se">\</span>
<span class="nt">--storagectl</span> SATA <span class="nt">--port</span> 0 <span class="nt">--type</span> hdd <span class="se">\</span>
<span class="nt">--discard</span> on <span class="se">\</span>
<span class="nt">--medium</span> <span class="s1">'<path/to/xxx.vdi>'</span>
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">--discard on</code> option enables discard support. However, the document says this feature is only supported with VDI disk file format. First, we cannot use raw block device for VDI disk file. Second, the VDI file’s allocation unit is 1MB, only when all disk pages in same MB boundary are <code class="language-plaintext highlighter-rouge">unmap</code>ed, the space should be released. NTFS allocates disk space in 4K pages by default. I succeed to change the allocation unit of NTFS to 1MB on Windows VM installation: just open a command prompt before the installation start with <code class="language-plaintext highlighter-rouge">Shift-F10</code>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>diskpart
> select disk 0
> select partition 2
> format fs=ntfs quick unit=1024K
> exit
exit
</code></pre></div></div>
<p>and then continue with the installation. However, I experience lock up frequently, the VM would hang and the VirtualBox process would begin to read from disk at around 300MB/s endlessly, then I have to reboot the host OS. I found a 3-years-old bug report in https://www.virtualbox.org/ticket/16450 for it, and the bug is still here 3 years later. Trying to get VirtualBox to work wasted me a LOT of time, since each attempt involves a reboot. And It is still not working.</p>
<h1 id="vmware-workstation-player">vmware workstation player</h1>
<p>So there is only one choice left, the vmware workstation player. It does provide good graphics performance over all the other VM softwares. The only shortcoming is no support for passing <code class="language-plaintext highlighter-rouge">unmap</code> to the raw block device.</p>
<p>As a workaround, I found a way to create a ‘dual bootable’ Windows VM: it could boot under both vmware workstation player and qemu. When a trimming is needed, I could boot it with qemu and do the trimming.</p>
<p>Since the <code class="language-plaintext highlighter-rouge">discard=unmap</code> option is only supported in <code class="language-plaintext highlighter-rouge">virtio-scsi</code> driver when using qemu, the Windows VM needs to support virtio device on early boot. The easier way is, install the Windows VM in qemu first (so that the virtio driver would be emebed in the Windows’ ‘initramfs’), then boot from the same raw block device in vmware workstation player and install vmware tools and drivers. Of course with dark magics from Windows, one could install the Windows VM in vmware first, and then install virtio driver in qemu (https://superuser.com/questions/1057959/windows-10-in-kvm-change-boot-disk-to-virtio/1095112). But doing the qemu part first makes life MUCH easier.</p>
<p>Please note that, vmware worksatation player boots VMs in efi mode by default, while qemu boots VMs in bios mode by default. I choose to let vmware boots in bios mode. Changing the <code class="language-plaintext highlighter-rouge">firmware = "efi"</code> line to <code class="language-plaintext highlighter-rouge">firmware = "bios"</code> in the vmware VM’s <code class="language-plaintext highlighter-rouge">*.vmx</code> file does the trick.</p>
<p>I use zfs on my laptop. I create a zfs sparse zvol with 128KB block size:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>zfs create <span class="nt">-V</span> 64G <span class="nt">-s</span> <span class="nt">-b</span> <span class="k">$((</span><span class="m">128</span> <span class="o">*</span> <span class="m">1024</span><span class="k">))</span> <span class="nt">-o</span> <span class="nv">primarycache</span><span class="o">=</span>metadata <some pool>/win10
<span class="nb">sudo </span>setfacl <span class="nt">-m</span> u:<user name>:rw /dev/zvol/<some pool>/win10
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">-s</code> stands for sparse. The <code class="language-plaintext highlighter-rouge">-b</code> option set the block size. The <code class="language-plaintext highlighter-rouge">-o primarycache=metadata</code> avoids zfs’ ARC cache the VM’s disk inside host memory. The <code class="language-plaintext highlighter-rouge">setfacl</code> changes ACL to the volume, so that accessing the volume does not require root permission. After formating the VM’s NTFS with 128K allocation unit, the zfs’ lz4 compression works great for the volume (hopefully it should perform better than NTFS’ compression).</p>
<h1 id="conclusion">Conclusion</h1>
<p>Finally, I got a Windows VM with acceptable graphics performance under GNU/Linux with <code class="language-plaintext highlighter-rouge">unmap</code> support. Since the graphics driver inside the guest VM does not provide hardware video decoding, the CPU usage during video playback is higher than native Windows machine. But we have lived without hardware video decoding in web browsers under GNU/Linux for a long time, don’t we?</p>Recently, I have to use some on-line meeting softwares whose desktop version support only Windows. So that once again, I need to install a Windows virtual machine on my laptop running Debian. I used to use qemu with kvm which deliver great IO performance with virtio drivers. However, its graphics performance is unbearable, even with the spice protocol and the qxl driver installed. I need to find a better Windows VM solution for GNU/Linux. Actually, there are lots of good VM softwares for GNU/Linux, for example, VirtualBox, qemu with kvm, and vmware series.LUKS Encrypted Root with GPG Smartcard2018-11-16T07:28:47+00:002018-11-16T07:28:47+00:00https://blog.fugoes.xyz/2018/11/16/LUKS-Encrypted-Root-with-GPG-Smartcard<h1 id="update-20190622">Update: 2019/06/22</h1>
<p>After upgrade to Debian 10 with <code class="language-plaintext highlighter-rouge">gpg 2.2</code>, the script is broken. One possible workaround is avoiding the use of <code class="language-plaintext highlighter-rouge">pinentry-curses</code>, which could be achieved by altering the content of the file <code class="language-plaintext highlighter-rouge">/etc/luks_gpg/decrypt.sh</code> to</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/sh</span>
<span class="nv">UUID</span><span class="o">=</span><span class="sb">`</span><span class="nb">basename</span> <span class="nv">$0</span><span class="sb">`</span>
<span class="nb">export </span><span class="nv">GNUPGHOME</span><span class="o">=</span>/etc/luks_gpg/
<span class="nb">read</span> <span class="nt">-p</span> <span class="s2">"pincode: "</span> <span class="nt">-s</span> pincode
<span class="nb">echo</span> <span class="s2">"</span><span class="nv">$pincode</span><span class="s2">"</span> | gpg <span class="nt">--batch</span> <span class="nt">--pinentry-mode</span> loopback <span class="nt">--passphrase-fd</span> 0 <span class="se">\</span>
<span class="nt">--no-tty</span> <span class="nt">--decrypt</span> <span class="s2">"/etc/luks_gpg/</span><span class="nv">$UUID</span><span class="s2">.key.gpg"</span>
</code></pre></div></div>
<p>and appending <code class="language-plaintext highlighter-rouge">allow-loopback-pinentry</code> option to <code class="language-plaintext highlighter-rouge">/etc/luks_gpg/gpg-agent.conf</code>.</p>
<hr />
<h1 id="background">Background</h1>
<p>I used to protect LUKS root on my Linux machines with a static password stored in my yubikey. It is not safe if losing my yubikey, since the static password does not require any pin code. I was always thinking about using yubikey’s GPG smartcard capacity to make it safer, and finally I spent about 1 hour getting it work today. XD</p>
<p>I generally followed this <a href="https://wiki.majic.rs/Openpgp/protecting_luks_decryption_key_in_debian_jessie_us">link</a>.</p>
<h1 id="how-to">How to</h1>
<p>I use Debian stretch, and I have already setup an LUKS partition on a partition whose <code class="language-plaintext highlighter-rouge">UUID</code> is <code class="language-plaintext highlighter-rouge"><UUID></code>. Login as <code class="language-plaintext highlighter-rouge">root</code> user, and follow these steps.</p>
<ol>
<li>Install necessary packages.
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>apt <span class="nb">install</span> <span class="nt">-y</span> scdaemon pinentry-curses
</code></pre></div> </div>
</li>
<li>Create <code class="language-plaintext highlighter-rouge">/etc/luks_gpg/</code> and set permission properly.
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">mkdir</span> <span class="nt">-p</span> /etc/luks_gpg
<span class="nb">chown </span>root:root /etc/luks_gpg
<span class="nb">chmod </span>700 /etc/luks_gpg
</code></pre></div> </div>
</li>
<li>Edit <code class="language-plaintext highlighter-rouge">/etc/luks_gpg/gpg-agent.conf</code> to force gpg to use curses interface since initramfs has no GUI.
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pinentry-program /usr/bin/pinentry-curses
</code></pre></div> </div>
</li>
<li>Import your public key with <code class="language-plaintext highlighter-rouge">$GNUPGHOME</code> set to <code class="language-plaintext highlighter-rouge">/etc/luks_gpg/</code>, and trust it.
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gpg <span class="nt">--homedir</span> /etc/luks_gpg <span class="nt">--import</span> <path_to_your_public_key>
gpg <span class="nt">--homedir</span> /etc/luks_gpg <span class="nt">--edit-key</span> you@example.com
gpg> trust
</code></pre></div> </div>
</li>
<li>Generate a password file for LUKS and setup it as LUKS’s password.
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">dd </span><span class="k">if</span><span class="o">=</span>/dev/random <span class="nv">of</span><span class="o">=</span>/etc/luks_gpg/<UUID>.key <span class="nv">bs</span><span class="o">=</span>1 <span class="nv">count</span><span class="o">=</span>256
cryptsetup luksAddKey /dev/disk/by-uuid/<UUID> /etc/luks_gpg/<UUID>.key
</code></pre></div> </div>
</li>
<li>Use your public key to encrypt the key and safely delete it.
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gpg <span class="nt">--homedir</span> /etc/luks_gpg <span class="nt">--encrypt</span> <span class="nt">--recipient</span> you@example.com /etc/luks_gpg/<UUID>.key
<span class="nb">shred</span> <span class="nt">-u</span> /etc/luks_gpg/<UUID>.key
</code></pre></div> </div>
</li>
<li>Add <code class="language-plaintext highlighter-rouge">/etc/luks_gpg/decrypt.sh</code> as decryption script.
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>#!/bin/sh
UUID=`basename $0`
export GNUPGHOME=/etc/luks_gpg/
gpg --no-tty --decrypt "/etc/luks_gpg/$UUID.key.gpg"
</code></pre></div> </div>
</li>
<li>Don’t forget to setup permission for it.
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">chmod</span> +x /etc/luks_gpg/decrypt.sh
</code></pre></div> </div>
</li>
<li>I applied a simple trick so that I could use different password for different partition easily, for each parition with different <code class="language-plaintext highlighter-rouge"><UUID></code>, we could create a soft link <code class="language-plaintext highlighter-rouge">/etc/luks_gpg/<UUID></code> to <code class="language-plaintext highlighter-rouge">/etc/luks_gpg/decrypt.sh</code>.
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cd</span> /etc/luks_gpg
<span class="nb">ln</span> <span class="nt">-s</span> decrypt.sh <UUID>
</code></pre></div> </div>
</li>
<li>Here is the trick. Update <code class="language-plaintext highlighter-rouge">/etc/crypttab</code> by appending the following to each LUKS partition with <code class="language-plaintext highlighter-rouge">UUID</code> of <code class="language-plaintext highlighter-rouge"><UUID></code>.
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>,keyscript=/etc/luks_gpg/<UUID>
</code></pre></div> </div>
</li>
<li>Add <code class="language-plaintext highlighter-rouge">/etc/initramfs-tools/hooks/luks_gpg</code> to include necessary binaries as well as <code class="language-plaintext highlighter-rouge">$GNUPGHOME</code> in initramfs.
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>#!/bin/sh
set -e
PREREQ="cryptroot"
prereqs()
{
echo "$PREREQ"
}
case $1 in
prereqs)
prereqs
exit 0
;;
esac
. /usr/share/initramfs-tools/hook-functions
cp -a /etc/luks_gpg/ "${DESTDIR}/etc/"
mkdir -p "${DESTDIR}/etc/terminfo/l/"
cp -a /lib/terminfo/l/linux "${DESTDIR}/etc/terminfo/l/linux"
copy_exec /usr/bin/gpg
copy_exec /usr/bin/gpg-agent
copy_exec /usr/bin/pinentry-curses
copy_exec /usr/lib/gnupg/scdaemon
exit 0
</code></pre></div> </div>
</li>
<li>Setup permissions for it.
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">chown </span>root:root /etc/initramfs-tools/hooks/luks_gpg
<span class="nb">chmod </span>750 /etc/initramfs-tools/hooks/luks_gpg
</code></pre></div> </div>
</li>
<li>Before updating the initramfs, you need to trigger a card-edit, or <code class="language-plaintext highlighter-rouge">scdaemon</code> would not be correctly triggered during boot.
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">export </span><span class="nv">GNUPGHOME</span><span class="o">=</span>/etc/luks_gpg/
gpg <span class="nt">--card-edit</span>
</code></pre></div> </div>
</li>
<li>Finally update the initramfs.
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>update-initramfs <span class="nt">-u</span>
</code></pre></div> </div>
</li>
</ol>
<h1 id="how-to-debug">How to debug</h1>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>update-initramfs -uv
lsinitramfs /boot/initrd.img-4.18.0-0.bpo.1-amd64
</code></pre></div></div>
<h1 id="links">Links</h1>
<ul>
<li><a href="https://wiki.majic.rs/Openpgp/protecting_luks_decryption_key_in_debian_jessie_us">https://wiki.majic.rs/Openpgp/protecting_luks_decryption_key_in_debian_jessie_us</a></li>
</ul>Update: 2019/06/22Chromebook 手记2018-09-19T10:21:47+00:002018-09-19T10:21:47+00:00https://blog.fugoes.xyz/2018/09/19/Notes-on-Chromebook<p>暑假刚开始的时候我就种草了 Chromebook ,而且恰在公司看到有人使用三星的 Chromebook Plus ,之后又拿 <a href="http://heroxbd.github.io/">heroxbd</a> 的 Chromebook Plus 玩了大半个月,终于决定购买它(然后发现汇率涨了好多)。于是我托去美国暑研的朋友帮忙购买,花去 410 刀,同时包装盒子由于不方便携带被朋友丢弃,不过据称包装里除去机器本身,仅有一个充电器和手写笔的替芯。 PD 充电器是 15V 30W 的(不要也罢)。</p>
<p>Chromebook Plus V1 主要的优点是:</p>
<ul>
<li>续航超长(出门一天无压力,配合上一个大一些的移动电源,几乎可以连续使用 24 小时,同时它的 PD 充电支持 5V ,如果是看视频的话,它的 Chrome browser 是支持硬解的);</li>
<li>屏幕极佳(400 nit 的 2K 屏幕,只要不到 3000 CNY);</li>
<li>有手写笔;</li>
<li>Hackable;</li>
<li>如果选择不太清真的 Chrome OS 的话,可以安装 Android APP ;</li>
<li>不贵。</li>
</ul>
<p>主要的缺点有:</p>
<ul>
<li>键盘比较糟糕(不过在长期在公司使用 MBP 2017 的键盘后,我觉得所有键盘都可以被原谅);</li>
<li>内置的 32 GB emmc 有点不够用,扩展 SD 卡有诸多限制;</li>
<li>4 GB 内存;</li>
<li>需要科学的网路才能使用。</li>
</ul>
<h2 id="android">Android</h2>
<p>使用不太清真的 Chrome OS 可以直接安装 Android APP(可以直接装 apk ,也可以用 Google Play) ,比较坑的是, Android APP 只能安装在 emmc 上,而且例如网易云音乐这种 APP 居然没法把文件存到 SD 卡上,于是放弃重度使用 Android 功能的想法。</p>
<h2 id="crouton">Crouton</h2>
<p><a href="https://github.com/dnschneid/crouton">crouton</a> 是一个基于 chroot 方案的跑 GNU Userland 的东西。为了节约宝贵的内置 emmc 空间,本人购买了一张 128 GB 的 SD 卡,此处一个坑是, Chrome OS 在锁屏时候会 umount 掉 SD 卡,解决方案是将 SD 分区,然后手动 mount 一个 ext4 分区到 <code class="language-plaintext highlighter-rouge">/usr/local/mnt/sdcard</code> 下,此时 Chrome OS 就不会自作聪明地 umount 掉它。</p>
<h3 id="安装">安装</h3>
<p>首先需要开启 Developer Mode ,<code class="language-plaintext highlighter-rouge">Ctrl+Alt+t</code> 可以打开一个 terminal , 下载好 <code class="language-plaintext highlighter-rouge">crouton</code> 文件之后:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>shell
<span class="nv">$ </span><span class="nb">cd</span> ~/Downloads
<span class="nv">$ </span><span class="nb">sudo </span>sh crouton <span class="nt">-p</span> /usr/local/mnt/sdcard/ <span class="se">\</span>
<span class="nt">-t</span> cli-extra <span class="se">\</span>
<span class="nt">-m</span> https://mirrors.tuna.tsinghua.edu.cn/debian/ <span class="se">\</span>
<span class="nt">-r</span> stretch <span class="se">\</span>
<span class="nt">-n</span> debian
</code></pre></div></div>
<p>然后使用 <code class="language-plaintext highlighter-rouge">sudo /usr/local/mnt/sdcard/bin/enter-chroot -n debian</code> 来使用它。</p>
<h3 id="cjk-问题">CJK 问题</h3>
<p>虽然 Chrome OS 支持各色输入法,但是默认的 terminal 对全宽字符的处理有 bug ,导致 emacs 和 vim 之类的在有中文的情况下几乎完全无法使用。我先后尝试了以下方案:</p>
<ol>
<li>套 tmux 和 screen :并没有什么用;</li>
<li>使用 secure shell 扩展:它和默认的 terminal 使用了相同的前端库,于是有相同的 bug ;</li>
<li>在安卓里使用 termux 之类的 APP ssh 回 Chrome OS :这些 terminal APP 中,输入法都无法使用;</li>
<li>使用 xiwi 开 xfce :试过了 fcitx 和 ibus ,都无法正常使用,而且 xiwi 不开高分支持辣眼睛,开了高分支持就很瞎眼;</li>
<li>使用 crostini :用不了输入法;</li>
<li>使用 web console :<a href="https://github.com/yudai/gotty">gotty</a> 非常科学!于是我终于有了一个可以正常使用中文的 <code class="language-plaintext highlighter-rouge">emacs-nox</code> 。。。</li>
</ol>
<p>现在我的做法是:切到 tty ,跑一个 tmux ,其中运行各种 daemon ,比如 gotty 的 daemon :</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>gotty <span class="nt">-a</span> 127.0.0.1 <span class="nt">-p</span> 8080 <span class="nt">-w</span> zsh
</code></pre></div></div>
<p>之后用 <code class="language-plaintext highlighter-rouge">vlock</code> 锁上这个 tmux ,这样就不需要在图形界面中为 chroot 留一个 tab 。</p>
<h3 id="gpg">gpg</h3>
<p>开箱即用,各种不涉及图形的功能都正常工作。</p>
<h2 id="crostini">Crostini</h2>
<p>目前的 beta channel 的 Chrome OS 可以使用 crostini 来运行 Linux 应用,但是这家伙实际上是个 kvm 。。。可以参考<a href="https://chromium.googlesource.com/chromiumos/platform/crosvm/">这里</a>,虽然 wayland 转发看上去非常厉害,但是这家伙实在太吃内存了,而 Chrome OS 默认使用 zswap ,于是开了 crostini 之后就会疯狂压缩解压缩,非常耗电,果断放弃使用。</p>
<h2 id="gentoo-prefix">gentoo prefix</h2>
<p>gentoo prefix 在 Chrome OS 中体验应该也很科学,不过我不太想 cross build aarch64 的 prefix 。。。</p>
<h2 id="其他">其他</h2>
<ul>
<li>可以把 <code class="language-plaintext highlighter-rouge">http://127.0.0.1:8080</code> pin 到 shelf ,从而按 <code class="language-plaintext highlighter-rouge">Alt+<number></code> 即可打开一个 terminal ;</li>
<li>各种 VPN 协议都有各种插件支持;</li>
<li>全屏之后 terminal 中的 <code class="language-plaintext highlighter-rouge">Ctrl-n</code> 之类的快捷键就不会被浏览器吃掉了;</li>
<li>可以设置比如 <code class="language-plaintext highlighter-rouge">Esc</code> 、 <code class="language-plaintext highlighter-rouge">Caps Lock</code> 等按键的功能,于是可以把 <code class="language-plaintext highlighter-rouge">Caps Lock</code> 和 <code class="language-plaintext highlighter-rouge">Esc</code> 互换位置。</li>
</ul>
<p>最后放一张图:
<img src="/assets/Chromebook 手记/screenshot.png" alt="emacs-nox" /></p>
<h1 id="links">Links</h1>
<ul>
<li><a href="http://scateu.me/2016/10/09/chromebook-rocks.html">http://scateu.me/2016/10/09/chromebook-rocks.html</a></li>
</ul>暑假刚开始的时候我就种草了 Chromebook ,而且恰在公司看到有人使用三星的 Chromebook Plus ,之后又拿 heroxbd 的 Chromebook Plus 玩了大半个月,终于决定购买它(然后发现汇率涨了好多)。于是我托去美国暑研的朋友帮忙购买,花去 410 刀,同时包装盒子由于不方便携带被朋友丢弃,不过据称包装里除去机器本身,仅有一个充电器和手写笔的替芯。 PD 充电器是 15V 30W 的(不要也罢)。Jeju Island2018-08-28T12:18:47+00:002018-08-28T12:18:47+00:00https://blog.fugoes.xyz/2018/08/28/Jeju-Island<p>看上去这是本人大学以后第一次出门旅游呢(其实是开会啦)。</p>
<p>飞机飞过天空:</p>
<p><img src="https://blog.fugoes.xyz/assets/Jeju Island/00.jpg" alt="飞机飞过天空" /></p>
<p>在飞驰的观光车上透过茶色玻璃所见:</p>
<p><img src="https://blog.fugoes.xyz/assets/Jeju Island/01.jpg" alt="茶色的滤镜" /></p>
<p><img src="https://blog.fugoes.xyz/assets/Jeju Island/02.jpg" alt="茶色的滤镜" /></p>
<p><img src="https://blog.fugoes.xyz/assets/Jeju Island/03.jpg" alt="茶色的滤镜" /></p>
<p>一朵路过了一座山的云:</p>
<p><img src="https://blog.fugoes.xyz/assets/Jeju Island/04.jpg" alt="山与云" /></p>
<p>阳光下清晰得刺眼:</p>
<p><img src="https://blog.fugoes.xyz/assets/Jeju Island/05.jpg" alt="山与海" /></p>
<p>海的那边是山:</p>
<p><img src="https://blog.fugoes.xyz/assets/Jeju Island/06.jpg" alt="海" /></p>
<p>海的那边还是海:</p>
<p><img src="https://blog.fugoes.xyz/assets/Jeju Island/07.jpg" alt="海" /></p>
<p>并不一色的水和天:</p>
<p><img src="https://blog.fugoes.xyz/assets/Jeju Island/08.jpg" alt="海" /></p>
<p>有些一色的水和天:</p>
<p><img src="https://blog.fugoes.xyz/assets/Jeju Island/09.jpg" alt="海" /></p>
<p>穿过桥的风:</p>
<p><img src="https://blog.fugoes.xyz/assets/Jeju Island/10.jpg" alt="桥" /></p>
<p>看不见的彩虹会消失吗:</p>
<p><img src="https://blog.fugoes.xyz/assets/Jeju Island/11.jpg" alt="彩虹" /></p>
<p>漏网的阳光:</p>
<p><img src="https://blog.fugoes.xyz/assets/Jeju Island/12.jpg" alt="阳光" /></p>
<p>MUA!</p>看上去这是本人大学以后第一次出门旅游呢(其实是开会啦)。Examples on LXC and KVM2018-04-07T06:00:00+00:002018-04-07T06:00:00+00:00https://blog.fugoes.xyz/2018/04/07/Examples-on-LXC-and-KVM<p>Recently I gave a talk on introduction to GNU/Linux, and I needed to provide each learner with a standalone GNU/Linux environment. So I created a KVM based virtual machine and ran tens of LXCs in it. This post notes down some commands. Using docker might be a better solution. However, I need to pay for the networking traffic consumption to download docker images, because my school haven’t provide mirror(or proxy) for docker images.</p>
<p>Create the KVM image:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>qemu-img create <span class="nt">-f</span> qcow2 debian.qcow2 32G
qemu-img create <span class="nt">-f</span> qcow2 lxc.qcow2 32G
</code></pre></div></div>
<p>Allocate the tap interface:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ip tuntap add dev tap0 mode tap user <span class="s2">"</span><span class="si">$(</span><span class="nb">whoami</span><span class="si">)</span><span class="s2">"</span>
</code></pre></div></div>
<p>If you want to do headless installation, you need to manually pass <code class="language-plaintext highlighter-rouge">console=ttyS0</code> to kernel:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>qemu-system-x86_64 <span class="se">\</span>
<span class="nt">-kernel</span> vmlinuz <span class="se">\</span>
<span class="nt">-initrd</span> initrd.gz <span class="se">\</span>
<span class="nt">-hda</span> debian.qcow2 <span class="se">\</span>
<span class="nt">-cdrom</span> debian-9.4.0-amd64-DVD-1.iso <span class="se">\</span>
<span class="nt">-append</span> <span class="s2">"console=ttyS0"</span> <span class="se">\</span>
<span class="nt">-nographic</span> <span class="se">\</span>
<span class="nt">-enable-kvm</span> <span class="se">\</span>
<span class="nt">-smp</span> 12 <span class="se">\</span>
<span class="nt">-m</span> <span class="k">$((</span><span class="m">1024</span> <span class="o">*</span> <span class="m">32</span><span class="k">))</span> <span class="se">\</span>
<span class="nt">-netdev</span> tap,ifname<span class="o">=</span>tap0,id<span class="o">=</span>tapnet <span class="se">\</span>
<span class="nt">-device</span> e1000,netdev<span class="o">=</span>tapnet
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">vmlinuz</code> and <code class="language-plaintext highlighter-rouge">initrd.gz</code> could be downloaded from debian’s mirrors, for example: https://mirrors.tuna.tsinghua.edu.cn/debian/dists/stable/main/installer-amd64/current/images/hd-media/ . After the installation, run the KVM with:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>qemu-system-x86_64 <span class="se">\</span>
<span class="nt">-hda</span> debian.qcow2 <span class="se">\</span>
<span class="nt">-hdb</span> lxc.qcow2 <span class="se">\</span>
<span class="nt">-nographic</span> <span class="se">\</span>
<span class="nt">-enable-kvm</span> <span class="se">\</span>
<span class="nt">-smp</span> 12 <span class="se">\</span>
<span class="nt">-m</span> <span class="k">$((</span><span class="m">1024</span> <span class="o">*</span> <span class="m">32</span><span class="k">))</span> <span class="se">\</span>
<span class="nt">-netdev</span> tap,ifname<span class="o">=</span>tap0,id<span class="o">=</span>tapnet <span class="se">\</span>
<span class="nt">-device</span> e1000,netdev<span class="o">=</span>tapnet
</code></pre></div></div>
<p>It should not need root permission, since the interface is allocated in advance. Now login to the KVM. You need to pass <code class="language-plaintext highlighter-rouge">console=ttyS0</code> to kernel to use tty by mounting the KVM image and editing <code class="language-plaintext highlighter-rouge">/boot/grub/grub.cfg</code>. Then setup LXC base image as <code class="language-plaintext highlighter-rouge">/var/lib/lxc/debian/</code> using <code class="language-plaintext highlighter-rouge">lxc-create</code>. You could create some simple scripts to create tens of overlay LXC instances. Here is an example configuration for overlay LXC images:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>lxc.network.type <span class="o">=</span> veth
lxc.network.flags <span class="o">=</span> up
lxc.network.link <span class="o">=</span> lxcbr0
lxc.include <span class="o">=</span> /usr/share/lxc/config/debian.common.conf
lxc.tty <span class="o">=</span> 4
lxc.arch <span class="o">=</span> amd64
lxc.rootfs <span class="o">=</span> aufs:/var/lib/lxc/debian/rootfs:/var/lib/lxc/debian-01/rootfs
lxc.utsname <span class="o">=</span> debian-01
lxc.network.hwaddr <span class="o">=</span> 00:FF:AA:00:00:01
</code></pre></div></div>
<p>Install <code class="language-plaintext highlighter-rouge">dnsmasq</code> and <code class="language-plaintext highlighter-rouge">aufs</code> in KVM (don’t forget to stop the <code class="language-plaintext highlighter-rouge">dnsmasq.service</code>, or the <code class="language-plaintext highlighter-rouge">dnsmasq</code> instance started by <code class="language-plaintext highlighter-rouge">lxc-net</code> would refuse to start), and doing some NAT, so that these LXCs could gain networking access. Note that <code class="language-plaintext highlighter-rouge">aufs</code> is not a block level copy-on-write file system, it may cause huge space usage if modifying huge files in the overlay LXC container.</p>
<p>To make a <code class="language-plaintext highlighter-rouge">qcow2</code> file smaller in file size:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>qemu-img convert <span class="nt">-O</span> qcow2 <span class="nt">-c</span> lxc.qcow2<span class="o">{</span>,_smaller<span class="o">}</span>
</code></pre></div></div>
<p>To attach a <code class="language-plaintext highlighter-rouge">qcow2</code> file:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>qemu-nbd <span class="nt">-c</span> /dev/nbd0 debian.qcow2
</code></pre></div></div>
<p>To detach it:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>qemu-nbd <span class="nt">-d</span> /dev/nbd0
</code></pre></div></div>Recently I gave a talk on introduction to GNU/Linux, and I needed to provide each learner with a standalone GNU/Linux environment. So I created a KVM based virtual machine and ran tens of LXCs in it. This post notes down some commands. Using docker might be a better solution. However, I need to pay for the networking traffic consumption to download docker images, because my school haven’t provide mirror(or proxy) for docker images.YubiKey 42018-04-05T02:00:00+00:002018-04-05T02:00:00+00:00https://blog.fugoes.xyz/2018/04/05/YubiKey-4<h1 id="introduction">Introduction</h1>
<p>According to <a href="https://en.wikipedia.org/wiki/YubiKey">wikipedia</a>:</p>
<blockquote>
<p>The YubiKey is a hardware authentication device manufactured by Yubico that supports one-time passwords, public key encryption and authentication, and the Universal 2nd Factor (U2F) protocol developed by the FIDO Alliance (FIDO U2F). It allows users to securely log into their accounts by emitting one-time passwords or using a FIDO-based public/private key pair generated by the device. YubiKey also allows for storing static passwords for use at sites that do not support one-time passwords. Facebook uses YubiKey for employee credentials, and Google supports it for both employees and users. Some password managers support YubiKey.</p>
</blockquote>
<h1 id="pam-capability">PAM capability</h1>
<p>For Debian:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>apt <span class="nb">install</span> <span class="nt">-y</span> libpam-u2f
<span class="nb">mkdir</span> <span class="nt">-p</span> ~/.config/Yubico
pamu2fcfg <span class="nt">-u</span><span class="s2">"</span><span class="si">$(</span><span class="nb">whoami</span><span class="si">)</span><span class="s2">"</span> <span class="o">></span> ~/.config/Yubico/u2f_keys
man pam_u2f
</code></pre></div></div>
<p>Then edit <code class="language-plaintext highlighter-rouge">/etc/pam.d/common-auth</code>:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>...
auth sufficient pam_u2f.so cue
auth <span class="o">[</span><span class="nv">success</span><span class="o">=</span>1 <span class="nv">default</span><span class="o">=</span>ignore] pam_unix.so nullok_secure try_first_pass
...
</code></pre></div></div>
<p>The above configuration would allow using either password or the yubikey to do authentication. The <code class="language-plaintext highlighter-rouge">cue</code> option would prompt a message to remind to touch the device. However the prompt is written to <code class="language-plaintext highlighter-rouge">stdout</code> instead of <code class="language-plaintext highlighter-rouge">stderr</code>.</p>
<h1 id="piv-capability">PIV capability</h1>
<p>I use <code class="language-plaintext highlighter-rouge">i3lock</code> as lock screen program. It use <code class="language-plaintext highlighter-rouge">pam</code> so everything works just fine. And I wrote some scripts to lock screen when unplugging yubikey.</p>
<p>The <code class="language-plaintext highlighter-rouge">/etc/udev/rules.d/70-yubikey.rules</code> file:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">ACTION</span><span class="o">==</span><span class="s2">"remove"</span>, <span class="nv">SUBSYSTEM</span><span class="o">==</span><span class="s2">"usb"</span>, ENV<span class="o">{</span>ID_VENDOR_ID<span class="o">}==</span><span class="s2">"1050"</span>, ENV<span class="o">{</span>ID_MODEL_ID<span class="o">}==</span><span class="s2">"0407"</span>, RUN+<span class="o">=</span><span class="s2">"/bin/bash /home/fugoes/.dotfiles/bin/usb-remove.sh"</span>
</code></pre></div></div>
<p>If the udev rules file is a softlink, you need to manually tirgger a reload with <code class="language-plaintext highlighter-rouge">udevadm control --reload</code>, and with every reboot, a reload is also needed.</p>
<p>The <code class="language-plaintext highlighter-rouge">/home/fugoes/.dotfiles/bin/usb-remove.sh</code> file (a dirty hack…):</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/bash</span>
<span class="nb">export </span><span class="nv">DISPLAY</span><span class="o">=</span>:0.0
<span class="nb">export </span><span class="nv">XAUTHORITY</span><span class="o">=</span><span class="s2">"/home/fugoes/.Xauthority"</span>
is_shift_down<span class="o">()</span> <span class="o">{</span>
xinput <span class="nt">--list</span>|
<span class="nb">grep</span> <span class="s2">"slave *keyboard"</span>|
<span class="nb">sed</span> <span class="nt">-r</span> <span class="s1">'s/.*id=([[:digit:]]+).*/\1/'</span>|
<span class="o">{</span>
<span class="k">while </span><span class="nb">read </span>num<span class="p">;</span> <span class="k">do
if </span>xinput <span class="nt">--query-state</span> <span class="s2">"</span><span class="nv">$num</span><span class="s2">"</span>|grep <span class="nt">-q</span> <span class="s1">'key\[50\]=down'</span><span class="p">;</span> <span class="k">then
</span><span class="nb">exit </span>0
<span class="k">fi
done
</span><span class="nb">exit </span>1
<span class="o">}</span>
<span class="o">}</span>
is_shift_down <span class="o">||</span> systemctl <span class="nt">--no-block</span> start i3lock.service
</code></pre></div></div>
<p>If you unplug yubikey with left shift key down, the screen lock would not be triggered. What’s more, DO NOT run daemon or long task with udev rules. It would kill child process after the script executed.</p>
<h1 id="gpg-smartcard-capability">GPG smartcard capability</h1>
<p>YubiKey 4 could be used as a GPG smartcard which could store 4096bit RSA private keys. <a href="https://github.com/drduh/YubiKey-Guide">Here</a> is a great guide. Notice that, <code class="language-plaintext highlighter-rouge">keytocard</code> would move the key to the card, and if you save after this command, your private keys which have been moved to the card in <code class="language-plaintext highlighter-rouge">GPGHOME</code> would be deleted. Please do backup before <code class="language-plaintext highlighter-rouge">keytocard</code>!</p>
<h1 id="u2f-capability">U2F capability</h1>
<p>Firefox do have native support for U2F now. Just enable <code class="language-plaintext highlighter-rouge">security.webauth.u2f</code> in <code class="language-plaintext highlighter-rouge">about:config</code>. And google’s products now support u2f functionality in Firefox.</p>
<h1 id="otp-capability">OTP capability</h1>
<p>TODO</p>
<h1 id="references">References</h1>
<ol>
<li><a href="https://en.wikipedia.org/wiki/YubiKey">https://en.wikipedia.org/wiki/YubiKey</a></li>
<li><a href="https://bigeagle.me/2016/02/yubikey-4">https://bigeagle.me/2016/02/yubikey-4</a></li>
<li><a href="https://github.com/drduh/YubiKey-Guide">https://github.com/drduh/YubiKey-Guide</a></li>
</ol>IntroductionRun Babeld over Wireguard2018-02-03T03:38:00+00:002018-02-03T03:38:00+00:00https://blog.fugoes.xyz/2018/02/03/Run-Babeld-over-Wireguard<p><a href="https://github.com/jech/babeld">Babeld</a> is a loop-avoiding distance-vector routing protocol. It could auto update link cost according to network delay, which is different from other distance-vector routing protocols like RIP. This unique feature makes it useful for personal overlay network. <a href="https://www.wireguard.com/">Wireguard</a> is a simple and fast layer 3 VPN software.</p>
<p>Wireguard doesn’t support IPv6 multicast, which is used by Babeld to do neighbor discovery. Wireguard forwards packets by matching <code class="language-plaintext highlighter-rouge">allowed-ips</code> option for its node. For a central node which other nodes connect to, you could decide which peer to forward a packet to by including the packet’s destination IPv6 address in the peer’s <code class="language-plaintext highlighter-rouge">allowed-ips</code>. These <code class="language-plaintext highlighter-rouge">allowed-ips</code> works like some kind of routing table. However, <code class="language-plaintext highlighter-rouge">allowed-ips</code> for different peers cannot overlap, which means you can not assign one address to multiple peers’ <code class="language-plaintext highlighter-rouge">allowed-ips</code>. As a result, you could only assign the multicast IPv6 address to one peer, and other peers would never receive <code class="language-plaintext highlighter-rouge">Babeld Hello</code> message from this central node. So you could only use Wireguard as a point-to-point layer 3 tunnel to run Babeld over it, or Babeld would fail to discover all links. As a result, for a n nodes full mesh network, each nodes need a Wireguard tunnel to all other n - 1 nodes.</p>
<p>By default, Wireguard would not assign IPv6 link local address to interface. In fact, Wireguard’s interface doesn’t have MAC address since it is an layer 3 VPN software. However, Babeld requires a IPv6 link local address to work. I use the following code to generate random IPv6 link local address for such interface to make Babeld happy:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">random</span>
<span class="k">def</span> <span class="nf">random_mac</span><span class="p">():</span>
<span class="n">digits</span> <span class="o">=</span> <span class="p">[</span><span class="mh">0x00</span><span class="p">,</span> <span class="mh">0x16</span><span class="p">,</span> <span class="mh">0x3e</span><span class="p">,</span> <span class="n">random</span><span class="p">.</span><span class="n">randint</span><span class="p">(</span><span class="mh">0x00</span><span class="p">,</span> <span class="mh">0x7f</span><span class="p">),</span> <span class="n">random</span><span class="p">.</span><span class="n">randint</span><span class="p">(</span><span class="mh">0x00</span><span class="p">,</span> <span class="mh">0xff</span><span class="p">),</span> <span class="n">random</span><span class="p">.</span><span class="n">randint</span><span class="p">(</span><span class="mh">0x00</span><span class="p">,</span> <span class="mh">0xff</span><span class="p">)]</span>
<span class="k">return</span> <span class="s">":"</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="s">"%02x"</span> <span class="o">%</span> <span class="n">x</span><span class="p">,</span> <span class="n">digits</span><span class="p">))</span>
<span class="k">def</span> <span class="nf">mac_to_ipv6</span><span class="p">(</span><span class="n">mac</span><span class="p">):</span>
<span class="n">parts</span> <span class="o">=</span> <span class="n">mac</span><span class="p">.</span><span class="n">split</span><span class="p">(</span><span class="s">":"</span><span class="p">)</span>
<span class="n">parts</span><span class="p">.</span><span class="n">insert</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="s">"ff"</span><span class="p">)</span>
<span class="n">parts</span><span class="p">.</span><span class="n">insert</span><span class="p">(</span><span class="mi">4</span><span class="p">,</span> <span class="s">"fe"</span><span class="p">)</span>
<span class="n">parts</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="s">"%x"</span> <span class="o">%</span> <span class="p">(</span><span class="nb">int</span><span class="p">(</span><span class="n">parts</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="mi">16</span><span class="p">)</span> <span class="o">^</span> <span class="mi">2</span><span class="p">)</span>
<span class="n">ipv6_parts</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">parts</span><span class="p">),</span> <span class="mi">2</span><span class="p">):</span>
<span class="n">ipv6_parts</span><span class="p">.</span><span class="n">append</span><span class="p">(</span><span class="s">""</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="n">parts</span><span class="p">[</span><span class="n">i</span><span class="p">:</span><span class="n">i</span> <span class="o">+</span> <span class="mi">2</span><span class="p">]))</span>
<span class="k">return</span> <span class="s">"fe80::%s/64"</span> <span class="o">%</span> <span class="p">(</span><span class="s">":"</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="n">ipv6_parts</span><span class="p">))</span>
<span class="k">def</span> <span class="nf">random_ipv6</span><span class="p">():</span>
<span class="k">return</span> <span class="n">mac_to_ipv6</span><span class="p">(</span><span class="n">random_mac</span><span class="p">())</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s">"__main__"</span><span class="p">:</span>
<span class="k">print</span><span class="p">(</span><span class="n">random_ipv6</span><span class="p">())</span>
</code></pre></div></div>
<p>Then just add these random generated IPv6 link local address to all interface(as well as normal IPv4 addresses) and set both ends of a Wireguard tunnel’s <code class="language-plaintext highlighter-rouge">allowed-ips</code> to <code class="language-plaintext highlighter-rouge">0.0.0.0/0,::/0</code>, which simply allow everything to go through the tunnel. Finally, start Babeld on all the nodes and all the interface with your configuration, it shall work.</p>Babeld is a loop-avoiding distance-vector routing protocol. It could auto update link cost according to network delay, which is different from other distance-vector routing protocols like RIP. This unique feature makes it useful for personal overlay network. Wireguard is a simple and fast layer 3 VPN software.Manual Bootstrap Gentoo Prefix for aarch64 Android2018-02-01T14:18:00+00:002018-02-01T14:18:00+00:00https://blog.fugoes.xyz/2018/02/01/Manual-Bootstrap-Gentoo-Prefix-for-aarch64-Android<h2 id="overview">Overview</h2>
<p>There is a helper script to aid bootstrapping Gentoo Prefix on different platforms. Please browse <a href="https://wiki.gentoo.org/wiki/Project:Prefix">this wiki page</a> to get this script. This <code class="language-plaintext highlighter-rouge">bootstrap-prefix.sh</code> works for arm64 platform with some manual tuning, due to the absent of <code class="language-plaintext highlighter-rouge">arm64/prefix</code> profile. After <a href="https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=4979fb71526b7919d218dededaf1c8803d201573">this commit</a>, it shall work smoothly. This script divides the whole process into three stages, namely stage1, stage2, and stage3. Stage1 and stage2 will bring you a tool chain in <code class="language-plaintext highlighter-rouge">${EPREFIX}/tmp</code>, as well as a <code class="language-plaintext highlighter-rouge">portage</code> and its dependencies. Stage3 will using the tool chain in <code class="language-plaintext highlighter-rouge">${EPREFIX}/tmp</code> to bootstrap a complete system.</p>
<h2 id="preparation">Preparation</h2>
<p>You need an native tool chain as well as other programs like <code class="language-plaintext highlighter-rouge">bash</code> to run the script. There are ways to achieve these requirements.</p>
<h3 id="method-0-using-an-aarch64-gentoo-prefix">Method 0: Using an aarch64 Gentoo prefix</h3>
<p>Yes, Gentoo prefix on Linux(RAP) could bootstrap itself to a different prefix path. This method is less time consuming than the following two, since you could directly go to stage3 without doing stage1 & 2 with an existing Gentoo prefix installed.</p>
<h3 id="method-1-using-a-native-gnulinux-environment">Method 1: Using a native GNU/Linux environment</h3>
<p>You could choose any Linux Distro that support aarch64 and install it on some device(Raspberry Pi 3 is a option).</p>
<p>The problem with this method is that it is hard to find powerful aarch64 machine running GNU/Linux. Using <code class="language-plaintext highlighter-rouge">chroot</code> with an Android device, or <code class="language-plaintext highlighter-rouge">qemu-chroot</code> with an amd64 GNU/Linux are acceptable workaround. After <code class="language-plaintext highlighter-rouge">chroot</code> into an aarch64 GNU/Linux, you could easily obtain a working tool chain with the package manager. Note that, <code class="language-plaintext highlighter-rouge">qemu-chroot</code>’s performance is not so good, though still acceptable. Generally speaking, a normal laptop’s performance under <code class="language-plaintext highlighter-rouge">qemu-chroot</code> is worse than a recent Android phone. What’s more, <code class="language-plaintext highlighter-rouge">qemu-chroot</code> doesn’t support all system calls, I encountered some errors when trying to bootstrap a stage3 Gentoo prefix with <code class="language-plaintext highlighter-rouge">qemu-chroot</code>, while stage1 & stage2 compiles happily with <code class="language-plaintext highlighter-rouge">qemu-chroot</code>, and you could copy stage2 to you Android device and doing stage3 there.</p>
<h3 id="method-2-cross-compile-an-aarch64-tool-chain">Method 2: Cross compile an aarch64 tool chain</h3>
<p>There is an outdated <a href="https://wiki.gentoo.org/wiki/Project:Android/build">wiki</a>. Note that, current cross <code class="language-plaintext highlighter-rouge">emerge</code> command ignores <code class="language-plaintext highlighter-rouge">EPREFIX</code> in its environment, but offers an command line options <code class="language-plaintext highlighter-rouge">--prefix</code> for it. I haven’t succeeded with this method.</p>
<h2 id="stage1--2">Stage1 & 2</h2>
<p>Before compiling everything, you might want to read <a href="https://wiki.gentoo.org/wiki/Project:Android/tarball">this wiki page</a>.</p>
<p>According to <a href="https://wiki.gentoo.org/wiki/Project:Prefix/Manual_Bootstrap">this wiki page</a>(It worths reading):</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">export </span><span class="nv">EPREFIX</span><span class="o">=</span><span class="s2">"/data/gentoo-arm64"</span>
<span class="nv">$ </span><span class="nb">export </span><span class="nv">PATH</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">EPREFIX</span><span class="k">}</span><span class="s2">/usr/bin:</span><span class="k">${</span><span class="nv">EPREFIX</span><span class="k">}</span><span class="s2">/bin"</span>
<span class="nv">$ </span><span class="nb">export </span><span class="nv">PATH</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">PATH</span><span class="k">}</span><span class="s2">:</span><span class="k">${</span><span class="nv">EPREFIX</span><span class="k">}</span><span class="s2">/tmp/usr/bin:</span><span class="k">${</span><span class="nv">EPREFIX</span><span class="k">}</span><span class="s2">/tmp/bin:/usr/bin:/bin"</span>
<span class="nv">$ </span>./bootstrap-prefix.sh <span class="s2">"</span><span class="k">${</span><span class="nv">EPREFIX</span><span class="k">}</span><span class="s2">"</span> setup
<span class="nv">$ </span>./bootstrap-prefix.sh <span class="s2">"</span><span class="k">${</span><span class="nv">EPREFIX</span><span class="k">}</span><span class="s2">"</span> stage1
<span class="nv">$ </span>./bootstrap-prefix.sh <span class="s2">"</span><span class="k">${</span><span class="nv">EPREFIX</span><span class="k">}</span><span class="s2">"</span> stage2
</code></pre></div></div>
<p>If you are lucky enough, you shall run these commands without any errors. If you encounter some errors, try to fix it by reading build logs. You don’t need to delete all files between two trail, the script will handle it correctly. You might want to do a snapshot between commands as checkpoint or change portage configuration during the process.</p>
<p>Some of the problems I encountered are:</p>
<ol>
<li>
<p>Wrong <code class="language-plaintext highlighter-rouge">gid</code> for <code class="language-plaintext highlighter-rouge">/dev/pts</code> on Android and <code class="language-plaintext highlighter-rouge">glibc</code> refuse to compile, to fix it:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>/system/bin/mount <span class="nt">-t</span> devpts <span class="nt">-o</span> remount,gid<span class="o">=</span>5 devpts /dev/pts
</code></pre></div> </div>
</li>
<li>
<p><code class="language-plaintext highlighter-rouge">/bin/bash</code> not found on Android(or something like that), you could fix it by doing a simple soft linking.</p>
</li>
<li>
<p>Some package failed to merge since trying to installed files outside of <code class="language-plaintext highlighter-rouge">$EPREFIX/</code>. I encountered this problem when forcing a stable keywords by adding <code class="language-plaintext highlighter-rouge">ACCEPT_KEYWORDS="arm64 -~arm64"</code> to <code class="language-plaintext highlighter-rouge">make.conf</code>. To fix it:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">echo</span> <span class="s2">"sys-devel/gcc-config ~arm64"</span> <span class="o">>></span> <span class="s2">"</span><span class="k">${</span><span class="nv">EPREFIX</span><span class="k">}</span><span class="s2">/etc/portage/package.accept_keywords"</span>
</code></pre></div> </div>
<p>Or you could just stick to the default unstable keywords which works fine.</p>
</li>
<li>
<p><code class="language-plaintext highlighter-rouge">groupadd</code> and <code class="language-plaintext highlighter-rouge">useradd</code> not found, to workaround it:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">touch</span> <span class="s2">"</span><span class="k">${</span><span class="nv">EPREFIX</span><span class="k">}</span><span class="s2">/bin/groupadd"</span> <span class="o">&&</span> <span class="nb">chmod</span> +x <span class="s2">"</span><span class="k">${</span><span class="nv">EPREFIX</span><span class="k">}</span><span class="s2">/bin/groupadd"</span>
<span class="nv">$ </span><span class="nb">touch</span> <span class="s2">"</span><span class="k">${</span><span class="nv">EPREFIX</span><span class="k">}</span><span class="s2">/bin/useradd"</span> <span class="o">&&</span> <span class="nb">chmod</span> +x <span class="s2">"</span><span class="k">${</span><span class="nv">EPREFIX</span><span class="k">}</span><span class="s2">/bin/useradd"</span>
</code></pre></div> </div>
<p>Then manually editing <code class="language-plaintext highlighter-rouge">/etc/{shadow,group}</code> if necessary.</p>
</li>
<li>
<p><code class="language-plaintext highlighter-rouge">emerge</code> failed to download distfiles(e.g. due to <code class="language-plaintext highlighter-rouge">wget</code> not found), just download it else where and put it under portage’s <code class="language-plaintext highlighter-rouge">distfile/</code> directory.</p>
</li>
</ol>
<h2 id="stage3">Stage3</h2>
<p>If you use Method 0 and you still want to use the helper script to do stage3, you need to manually do necessary soft links under <code class="language-plaintext highlighter-rouge">/bin</code> and <code class="language-plaintext highlighter-rouge">${EPREFIX}/{bin,usr/bin}</code> for bash, python, etc. Then soft link the <code class="language-plaintext highlighter-rouge">usr/portage</code> directory in previous Gentoo prefix to the new one. Don’t forget to include the previous Gentoo prefix in <code class="language-plaintext highlighter-rouge">PATH</code>:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">export </span><span class="nv">PATH</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">EPREFIX</span><span class="k">}</span><span class="s2">/usr/bin:</span><span class="k">${</span><span class="nv">EPREFIX</span><span class="k">}</span><span class="s2">/bin"</span>
<span class="nv">$ </span><span class="nb">export </span><span class="nv">PATH</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">PATH</span><span class="k">}</span><span class="s2">:</span><span class="k">${</span><span class="nv">EPREFIX</span><span class="k">}</span><span class="s2">/tmp/usr/bin:</span><span class="k">${</span><span class="nv">EPREFIX</span><span class="k">}</span><span class="s2">/tmp/bin"</span>
<span class="nv">$ </span><span class="nb">export </span><span class="nv">PATH</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">PATH</span><span class="k">}</span><span class="s2">:</span><span class="k">${</span><span class="nv">PREVIOUS_EPREFIX</span><span class="k">}</span><span class="s2">/usr/bin:</span><span class="k">${</span><span class="nv">PREVIOUS_EPREFIX</span><span class="k">}</span><span class="s2">/bin"</span>
</code></pre></div></div>
<p>Note that, the sequence matters.</p>
<p>After copying stage2 to your Android device, you could use the helper script to bootstrap stage3:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>./bootstrap-prefix.sh <span class="s2">"</span><span class="k">${</span><span class="nv">EPREFIX</span><span class="k">}</span><span class="s2">"</span> stage3
</code></pre></div></div>
<p>You might encounter errors similar to what I listed in previous section, and the solutions are similar, too. Other than already listed errors, problems I encountered in this stage are:</p>
<ol>
<li>Failed to build <code class="language-plaintext highlighter-rouge">gmp</code> due to some C++ errors. Simply disable the <code class="language-plaintext highlighter-rouge">cxx</code> use flag for <code class="language-plaintext highlighter-rouge">gmp</code>, this use flag is not needed to build <code class="language-plaintext highlighter-rouge">gcc</code>.</li>
<li>Wrong shebang, fix it by manual editing the shebang or doing soft linking.</li>
</ol>
<h2 id="finishing">Finishing</h2>
<p>You shall do a <code class="language-plaintext highlighter-rouge">emerge -e @system</code> after finishing doing stage3. You could use <code class="language-plaintext highlighter-rouge">distcc</code> to make it less time consuming. And since you might encounter errors, while <code class="language-plaintext highlighter-rouge">emerge -e</code> will not remember which package you have already rebuild:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>emerge <span class="nt">-e</span> <span class="nt">-pv</span> @system <span class="o">></span> pkgs
</code></pre></div></div>
<p>Then edit <code class="language-plaintext highlighter-rouge">./pkgs</code> to get a package list. Do not delete packages’ version! Then:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">cat</span> ./pkgs|while <span class="nb">read </span>pkg<span class="p">;</span> <span class="k">do
</span>emerge <span class="nt">--oneshot</span> <span class="nt">--nodeps</span> <span class="s2">"</span><span class="k">${</span><span class="nv">pkg</span><span class="k">}</span><span class="s2">"</span> <span class="o">||</span> <span class="nb">exit </span>1
<span class="k">done</span>
</code></pre></div></div>
<h2 id="using-the-prefix">Using the prefix</h2>
<h3 id="script-to-start-prefix">Script to start prefix</h3>
<p>You might want to use a script to start the prefix. It is a bit more tricky for Android. Please refer to <a href="https://wiki.gentoo.org/wiki/Project:Android/tarball">this wiki page</a> for more information.</p>
<h3 id="ssh">SSH</h3>
<p>The <code class="language-plaintext highlighter-rouge">etc/init.d/sshd</code> script doesn’t work well with prefix. You might want to run <code class="language-plaintext highlighter-rouge">sshd</code> manually.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">HOME</span><span class="o">=</span>/data/gentoo-arm64/root /data/gentoo-arm64/usr/sbin/sshd <span class="nt">-E</span> sshd.log <span class="nt">-p</span> 22222
</code></pre></div></div>
<p>Put your public key in <code class="language-plaintext highlighter-rouge">/data/gentoo-arm64/root/.ssh/authorized_key</code> and change sshd’s configuration so that root user could login. If you are still not able to do ssh, try adding <code class="language-plaintext highlighter-rouge">StrictModes No</code> to the configuration file and restart the daemon.</p>
<h3 id="distcc">Distcc</h3>
<p>Distcc is a tool to distribute C family language compilation to speed things up. A general guide could be found <a href="https://wiki.gentoo.org/wiki/Distcc">here</a>. For this scenery, cross Distcc is needed. <a href="https://wiki.gentoo.org/wiki/Distcc/Cross-Compiling">This</a> is a guide for cross Distcc. I encountered some <strong>problems</strong> with the cross Distcc. The <code class="language-plaintext highlighter-rouge">${EPREFIX}/usr/lib/distcc/bin}</code> folder shall be <code class="language-plaintext highlighter-rouge">${EPREFIX}/usr/lib64/distcc/bin</code> for aarch64 devices.</p>
<p>Distcc’s plain mode would run the preprocessing on the local machine and the pump mode would try to run preprocessing remotely. Sometimes distcc pump mode might failed to build some package(because pump mode assumes headers will not change during the compilation, but for some package, e.g. <code class="language-plaintext highlighter-rouge">llvm</code>, this is not true), try using Distcc’s plain mode instead. You could still enable the <code class="language-plaintext highlighter-rouge">lzo</code> option with plain mode, which would reduce the the networking load.</p>Overview迁移到 Jekyll2017-06-28T13:25:00+00:002017-06-28T13:25:00+00:00https://blog.fugoes.xyz/2017/06/28/%E8%BF%81%E7%A7%BB%E5%88%B0-Jekyll<p>Hexo 又又又跪了,于是决定放弃 NodeJS 以及 npm 这些鬼畜的东西,换成基于 ruby 的 jekyll 。</p>
<h2 id="基本安装">基本安装</h2>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>aptitude <span class="nb">install </span>ruby ruby-dev rubygems
gem <span class="nb">install </span>jekyll bundler <span class="nt">--user-install</span>
<span class="c"># 注意将 gem 的目录加进 PATH</span>
jekyll new fugoes.github.io
<span class="nb">cd </span>fugoes.github.io
bundle <span class="nb">exec </span>jekyll serve
</code></pre></div></div>
<h2 id="转换-_posts">转换 _posts/</h2>
<p>来花式 <code class="language-plaintext highlighter-rouge">sed</code> 。。。</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cd </span>_posts
<span class="k">for </span>i <span class="k">in</span> <span class="k">*</span>.md <span class="nb">mv</span> <span class="o">{</span>,<span class="si">$(</span><span class="nb">cat</span> <span class="nv">$i</span>|grep <span class="s1">'^date: '</span>|awk <span class="s1">'{print $2}'</span>|sed <span class="nt">-r</span> <span class="s1">'s/$/-/'</span><span class="si">)</span><span class="o">}</span><span class="nv">$i</span>
<span class="k">for </span>i <span class="k">in</span> <span class="k">*</span>.md
<span class="k">do
</span><span class="nb">sed</span> <span class="nt">-i</span> <span class="nt">-r</span> <span class="s1">'s/^title: (.*)$/title: "\1"/'</span> <span class="nv">$i</span>
<span class="nb">sed</span> <span class="nt">-i</span> <span class="nt">-r</span> <span class="s1">'s/^date: (.*)$/date: \1 \+0800/'</span> <span class="nv">$i</span>
<span class="nb">sed</span> <span class="nt">-i</span> <span class="nt">-r</span> <span class="s1">'2ilayout: post'</span> <span class="nv">$i</span>
<span class="nb">sed</span> <span class="nt">-i</span> <span class="nt">-r</span> <span class="s1">'/<!--more-->/d'</span> <span class="nv">$i</span>
<span class="k">done</span>
</code></pre></div></div>
<p>就好了。。。</p>
<h2 id="其他配置">其他配置</h2>
<p>直接参考默认主题的 <a href="https://github.com/jekyll/minima">README</a> 。</p>
<h2 id="坑">坑</h2>
<h3 id="关于-gem">关于 <code class="language-plaintext highlighter-rouge">gem</code></h3>
<p><code class="language-plaintext highlighter-rouge">gem</code> 也给我一种鬼畜的感觉,因为我没有使用 <code class="language-plaintext highlighter-rouge">sudo</code> 运行它,它还是装了文件到 <code class="language-plaintext highlighter-rouge">/var/lib/gems</code> 中,我也是醉了。。。</p>
<h3 id="插图片">插图片</h3>
<p>将图片放到 <code class="language-plaintext highlighter-rouge">assets/</code> 下面和标题同名的目录下,然后使用<code class="language-plaintext highlighter-rouge">{{ site.url }}/assets/{{ page.title }}/picture01.jpg</code> 这样的方式来引用图片。</p>
<h3 id="hell-of-quotes">Hell of Quotes</h3>
<p>我的<a href="/2017/01/17/Blog-%E8%AE%B0%E4%BA%8B.html">这篇博客</a>里面有一段某模板语言的代码块,然后因为没有转义被 Jekyll 当成了模板去渲染了,于是手动在这段代码前后添加一对<code class="language-plaintext highlighter-rouge">{% raw %} {% endraw %}</code>,你也许想知道这句话是如何 escape 出来的,请参考<a href="https://stackoverflow.com/questions/3426182/how-to-escape-liquid-template-tags">这里</a>。。。</p>Hexo 又又又跪了,于是决定放弃 NodeJS 以及 npm 这些鬼畜的东西,换成基于 ruby 的 jekyll 。Play with tinc, iptables and dnsmasq2017-03-12T07:30:34+00:002017-03-12T07:30:34+00:00https://blog.fugoes.xyz/2017/03/12/Play-with-tinc-iptables-and-dnsmasq<p>看到 bigeagle 的<a href="https://bigeagle.me/2016/02/ipset-policy-routing/">这篇博客</a>,感觉非常棒,遂决定试一试,中途遇到了一些坑,好在都顺利解决了,遂记录之。(本文基于 Linux )</p>
<h1 id="目标">目标</h1>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> +----+
+----| do |----+
| +----+ |
| |
| |
| |
| |
+--+---+ +----+----+
| dell | | tencent |
+------+ +---------+
</code></pre></div></div>
<p>如图所示, do 是一台有公网 IP 的 VPS , tencent 是一台在(玄学) NAT 里的 VPS, dell 是我的笔记本,目标是所有机器都连接到 do ,并且 dell 可以直接访问(也就是不走 do ,直接穿透 NAT ) tencent 。</p>
<h1 id="step-1-setup-tinc-vpn">Step 1: Setup Tinc-VPN</h1>
<p>可以参考的<a href="https://cn2.chionlab.moe/2016/12/12/better-way-to-bypass-gfw-with-tinc/">一篇博客</a>的 <code class="language-plaintext highlighter-rouge">tinc 安装及配置</code> 部分。</p>
<h1 id="step-2-setup-tinc-as-gateway">Step 2: Setup Tinc as Gateway</h1>
<p>tinc 配置如下:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> +----+ do: Address = 192.168.100.1
+----| do |----+
| +----+ |
| | dell: Address = 192.168.100.2
| |
| |
| | tencent: Address = 192.168.100.3
+--+---+ +----+----+
| dell | | tencent |
+------+ +---------+
</code></pre></div></div>
<p>此时互相 ping Address 都可以 ping 通。</p>
<h2 id="server-side-setup">Server Side Setup</h2>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>iptables <span class="nt">-t</span> nat <span class="nt">-A</span> POSTROUTING <span class="nt">-s</span> 192.168.100.0/24 <span class="nt">-j</span> MASQUERADE <span class="nt">-o</span> eth0
</code></pre></div></div>
<p>至此, do 就可以作为 Gateway 来使用了。</p>
<h2 id="client-side-setup">Client Side Setup</h2>
<p>官方有一个<a href="http://tinc-vpn.org/examples/redirect-gateway/">栗子</a>,这个栗子其实已经基本满足需求了,但是有一个缺点就是需要手动设置一下 VPS 的路由,必须让目的地是 VPS 的包走外网而不是 tinc 的虚拟的 interface (我的 tinc 对应的 interface 是 flux),如果你用作 gateway 的那台机器在NAT里面,而且NAT最外面的公网 IP 还会变化,这种做法就变得不现实,在折腾了一番之后,我找到了一个有效的解决方案。具体的做法是,新建一个用户给 tinc 使用(比如 <code class="language-plaintext highlighter-rouge">sudo useradd tinc -s /bin/nologin</code> ),然后在 iptables 中通过 match uid 来实现给 tinc 用户的包打上 MARK ,再通过 ip rule 为这些包单独设置一个路由表。这样 tinc 就会处理好 NAT 以及 IP 的变化。</p>
<p>中途遇到的一个坑是, tinc 本身提供了一个 <code class="language-plaintext highlighter-rouge">--user=</code> 的选项,但是使用这个选项指定用户之后并不是预期的效果,发往 VPS 的公网 IP 的包会走错 interface ,我猜测的原因是, tinc 启动时会首先使用 root 权限新建一个 tun interface , 导致实际上的需要发往 VPS 公网 IP 的包的 uid 是 0 (也就是 root ) ,导致上面的做法失效,解决方法是,手动新建一个 uid 是 tinc 的 tun interface :</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>ip tuntap add dev flux mode tun user tinc group tinc
<span class="nb">sudo mknod</span> /dev/net/flux c 10 200
<span class="nb">sudo chown </span>tinc:tinc /dev/net/flux
</code></pre></div></div>
<p>然后:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>bash <span class="nt">-c</span> <span class="s2">"echo '' > /etc/tinc/flux/tinc-up"</span>
<span class="nb">sudo </span>bash <span class="nt">-c</span> <span class="s2">"echo '' > /etc/tinc/flux/tinc-down"</span>
</code></pre></div></div>
<p>注意修改一下 <code class="language-plaintext highlighter-rouge">/etc/tinc/flux/tinc.conf</code> :</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>...
Device = /dev/net/flux
Interface = flux
...
</code></pre></div></div>
<p>运行 tinc 时,使用 <code class="language-plaintext highlighter-rouge">sudo -u tinc tincd blablabla</code> ,然后一切就和预期的相同了。</p>
<p>具体的脚本见后文。</p>
<h1 id="step-3-setup-dnsmasq--ipset">Step 3: Setup Dnsmasq & ipset</h1>
<p>这一部分主要参考了<a href="https://bigeagle.me/2016/02/ipset-policy-routing/"> bigeagle 的博客</a>,不多赘述,最后综合上一步的方法以及大鹰的博客得到的一份脚本如下:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">cat</span> ~/bin/tinc
<span class="c">#!/bin/bash</span>
<span class="nv">dev</span><span class="o">=</span><span class="s2">"wlp7s0"</span>
<span class="nv">tinc_gateway</span><span class="o">=</span><span class="s2">"192.168.100.1"</span>
<span class="nv">tinc_user</span><span class="o">=</span><span class="s2">"tinc"</span>
<span class="nv">tinc_group</span><span class="o">=</span><span class="s2">"tinc"</span>
<span class="nv">INTERFACE</span><span class="o">=</span><span class="s2">"flux"</span>
up<span class="o">()</span> <span class="o">{</span>
<span class="k">if</span> <span class="o">[</span> <span class="nt">-a</span> /dev/net/<span class="nv">$INTERFACE</span> <span class="o">]</span><span class="p">;</span> <span class="k">then
</span><span class="nb">echo</span> <span class="s2">"File exist!"</span>
<span class="k">else
</span><span class="nb">echo</span> <span class="s2">"Create a nod!"</span>
<span class="nb">sudo mknod</span> /dev/net/<span class="nv">$INTERFACE</span> c 10 200
<span class="nb">sudo chown</span> <span class="nv">$tinc_user</span>:<span class="nv">$tinc_group</span> /dev/net/<span class="nv">$INTERFACE</span>
<span class="k">fi
</span><span class="nb">sudo </span>ip tuntap add dev <span class="nv">$INTERFACE</span> mode tun user <span class="nv">$tinc_user</span> group <span class="nv">$tinc_group</span>
<span class="nv">uid</span><span class="o">=</span><span class="si">$(</span><span class="nb">id</span> <span class="nt">-u</span> <span class="nv">$tinc_user</span><span class="si">)</span>
<span class="nb">sudo </span>iptables <span class="nt">-t</span> mangle <span class="nt">-N</span> DIRECT
<span class="nb">sudo </span>iptables <span class="nt">-t</span> mangle <span class="nt">-A</span> DIRECT <span class="nt">-m</span> owner <span class="nt">--uid-owner</span> <span class="nv">$uid</span> <span class="nt">-j</span> MARK <span class="nt">--set-mark</span> 42
<span class="nb">sudo </span>iptables <span class="nt">-t</span> mangle <span class="nt">-A</span> DIRECT <span class="nt">-m</span> <span class="nb">set</span> <span class="nt">--match-set</span> direct dst <span class="nt">-j</span> MARK <span class="nt">--set-mark</span> 42
<span class="nb">sudo </span>iptables <span class="nt">-t</span> mangle <span class="nt">-A</span> OUTPUT <span class="nt">-j</span> DIRECT
<span class="nb">sudo </span>iptables <span class="nt">-t</span> mangle <span class="nt">-A</span> POSTROUTING <span class="nt">-j</span> DIRECT
<span class="nb">sudo </span>iptables <span class="nt">-t</span> nat <span class="nt">-A</span> POSTROUTING <span class="nt">-o</span> <span class="nv">$dev</span> <span class="nt">-m</span> mark <span class="nt">--mark</span> 42 <span class="nt">-j</span> MASQUERADE
ip route show dev <span class="nv">$dev</span> | <span class="k">while </span><span class="nb">read </span>gwroute<span class="p">;</span> <span class="k">do
</span><span class="nb">sudo </span>ip route add <span class="nv">$gwroute</span> dev <span class="nv">$dev</span> table direct
<span class="k">done
</span><span class="nb">sudo </span>ip rule add fwmark 42 table direct
<span class="nb">sudo </span>ip <span class="nb">link set</span> <span class="nv">$INTERFACE</span> up
<span class="nb">sudo </span>ip addr add 192.168.100.2/32 dev <span class="nv">$INTERFACE</span>
<span class="nb">sudo </span>ip route add 192.168.100.0/24 dev <span class="nv">$INTERFACE</span>
<span class="c"># A small trick to preserve the default route</span>
<span class="nb">sudo </span>ip route add 0.0.0.0/1 dev <span class="nv">$INTERFACE</span>
<span class="nb">sudo </span>ip route add 128.0.0.0/1 dev <span class="nv">$INTERFACE</span>
<span class="nb">sudo mkdir</span> <span class="nt">-p</span> /var/run/tinc
<span class="nb">sudo chown</span> <span class="nt">-R</span> <span class="nv">$tinc_user</span>:<span class="nv">$tinc_group</span> /var/run/tinc
<span class="nb">sudo</span> <span class="nt">-u</span> <span class="nv">$tinc_user</span> tincd <span class="nt">-n</span> flux <span class="nt">-D</span> <span class="nt">--debug</span><span class="o">=</span>3 <span class="nt">--pidfile</span><span class="o">=</span>/var/run/tinc/<span class="nv">$INTERFACE</span>.pid
<span class="o">}</span>
down<span class="o">()</span> <span class="o">{</span>
<span class="nb">sudo kill</span> <span class="si">$(</span><span class="nb">cat</span> /var/run/tinc/<span class="nv">$INTERFACE</span>.pid<span class="si">)</span>
<span class="nb">sudo rm</span> <span class="nt">-rf</span> /var/run/tinc
<span class="nb">sudo </span>ip route del 0.0.0.0/1 dev <span class="nv">$INTERFACE</span>
<span class="nb">sudo </span>ip route del 128.0.0.0/1 dev <span class="nv">$INTERFACE</span>
<span class="nb">sudo </span>ip route del 192.168.100.0/24 dev <span class="nv">$INTERFACE</span>
<span class="nb">sudo </span>ip addr del 192.168.100.2/32 dev <span class="nv">$INTERFACE</span>
<span class="nb">sudo </span>ip <span class="nb">link set</span> <span class="nv">$INTERFACE</span> down
<span class="nb">sudo </span>ip rule del table direct
<span class="nb">sudo </span>ip route flush table direct
<span class="nb">sudo </span>iptables <span class="nt">-t</span> nat <span class="nt">-F</span>
<span class="nb">sudo </span>iptables <span class="nt">-t</span> mangle <span class="nt">-F</span>
<span class="nb">sudo </span>iptables <span class="nt">-t</span> mangle <span class="nt">-X</span> DIRECT
<span class="nb">sudo </span>ip tuntap del dev <span class="nv">$INTERFACE</span> mode tun
<span class="nb">sudo rm</span> /dev/net/<span class="nv">$INTERFACE</span>
<span class="o">}</span>
<span class="k">case</span> <span class="nv">$1</span> <span class="k">in
</span>up <span class="p">)</span>
up <span class="p">;;</span>
down <span class="p">)</span>
down <span class="p">;;</span>
<span class="k">esac</span>
</code></pre></div></div>
<p>将这个脚本丢到 PATH 里,然后使用 <code class="language-plaintext highlighter-rouge">tinc up</code> 启动代理,<code class="language-plaintext highlighter-rouge">tinc down</code> 关闭之。</p>
<p>可以添加到 systemd 开机启动:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">cat </span>tinc-flux.service
<span class="o">[</span>Unit]
<span class="nv">Description</span><span class="o">=</span>Tinc!
<span class="nv">After</span><span class="o">=</span>multi-user.target
<span class="o">[</span>Service]
<span class="nv">Type</span><span class="o">=</span>simple
<span class="nv">User</span><span class="o">=</span>fugoes
<span class="nv">ExecStart</span><span class="o">=</span>/home/fugoes/bin/tinc up
<span class="nv">ExecStop</span><span class="o">=</span>/home/fugoes/bin/tinc down
<span class="o">[</span>Install]
<span class="nv">WantedBy</span><span class="o">=</span>multi-user.target
</code></pre></div></div>
<p>注意到脚本中给为 direct table 配置的路由表是根据当前的默认路由生成的,所以换一个网络环境需要重启一下服务。</p>
<p>另外还有一个小 Tip 是:如果 ipset 不设置 timeout , 那个 table 就会越来越大,这当然是不科学的,所以修改一下 ipset 的配置:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">cat</span> /etc/ipset.conf
create direct <span class="nb">hash</span>:ip family inet hashsize 1024 maxelem 65536 <span class="nb">timeout </span>14400
add direct 166.111.8.28 <span class="nb">timeout </span>0
</code></pre></div></div>
<p>对于需要永久有效的 ipset 项目,设置 timeout 为 0 即可。</p>
<p>但是这样做会带来另一个问题,如果你的 timeout 设置的过小,小于某个 DNS 结果的 ttl ,那么 ipset 中的对应条目会在 timeout 秒之后失效,这个时候,由于 DNS 的 ttl 还没有结束,所以 dnsmasq 不会重新查询这个条目,于是 ipset 中就会有一段时间没有这个条目,解决方案也是简单的,配置一下 dnsmasq 即可:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>bash <span class="nt">-c</span> <span class="s1">'echo "max-cache-ttl=10800" >> /etc/dnsmasq.conf'</span>
</code></pre></div></div>
<p>只要保证这个 max-cache-ttl 的值大于 ipset 的 timeout ,在至多 max-cache-ttl 时间之后, dnsmasq 会更新一次 ipset 中对应的条目,该条目的 timeout 会被重置为默认值 (在我的栗子中是 14400 )。</p>
<p>PS. 本文中的服务端使用的是 debian testing ,客户端使用的是 Arch Linux ,不同发行版配置文件位置可能不同。每台电脑的网卡对应的 interface 名字不一定相同,自行修改。</p>看到 bigeagle 的这篇博客,感觉非常棒,遂决定试一试,中途遇到了一些坑,好在都顺利解决了,遂记录之。(本文基于 Linux )