Yi Tang Data Science and Emacs

Terminating Ethernet Cable At Height For CCTV Cameras

On the Ground

The standard Cat 6 plug is a pain to work with: I have to untwist the 4 pairs, make them perfectly straight, lay the 8 wires side by side with no gaps, and then insert all of them into the RJ45 plug in one go.

It sounds easy, but since the wires are flexible, it is actually very hard: very often the wires move around and become misaligned or misplaced during the fitting. If that happened or any other part of it went wrong, I would have to pull out the whole lot and restart again.

I had successes before, usually after a couple of attempts, often accompanied by frustration in between. It requires me to activate the fight mode, give it 100% focus while sitting in an “Orz” position1, so there’s quite a lot of energy poured into it.

At Height

However, even if I want to, it becomes physically impossible when it comes to fitting a plug in the air for the CCTV cameras: the ladder is a bit wobbly with uneven ground underneath it, and it is windy and raining due to a summer storm.

Since I wasn’t happy with the normal Cat 6 plug, I was keen to try new products. So when I first saw the IDC Punch Down to RJ45 Plug from Kenable 2, I ordered a few. It turned out to be a smart little move (this time).

This product has a built-in RJ45 plug that is already wired up, so I can skip that difficult part. All I have to do is punch down the wires into the IDC terminal. Punching down itself is very easy; I can do it half-minded with one hand.

Another benefit is that I can split the fitting into multiple steps, and I can take mini breaks for my arms between steps. Once one or two wires are inserted into the IDC terminal, it binds the cable to the plug. The binding is strong, so it hangs in the air and swings a bit with the wind with no issues. Then I take my time for the rest of the wires. If you don’t appreciate how important it is, trust me, your arms become rather fatigued when working with your hands overhead.

Footnotes

1 For people who don’t know what “Orz” stands for, “O” is head, “z” is legs and hips, and “r” is arms.

2 This post is not affiliated with Kenable

Extend Ethernet Cable

I had to extend the main Ethernet cable that connects the main router in the living room to the secondary router in the new office. Technically, the cable size is spot on, but I had to cut back 2-3 times because a combination of my lack of experience and the LAP data module from Screwfix is rubbish.

It seems like an unusual task given that there are only a few products available on the market. I tested two, and I am happy with the results, so I am documenting here for people who might find it useful.

Jelly Crimps

The first product I tested was from my electrician. It took me a while to find out that its name is Jelly Crimps. You can get it from TLC or Amazon.

The little connector has two long sleeves that host two wires. It has a button in the middle; press it very hard, and it will release the gel. I highly recommend using a piler unless you have super strong figures.

The process is simply: insert the wires, press with a piler to release the gel, and repeat 8 times for each wire.


Jelly Crimps in Use

It costs about £0.2 to extend one cable, so it is very cost-effective. I wasn’t sure it would work, but it does, and my electrician vouches for it.

The only problem with this product is that it is not maintenance-free. According to my electrician, I will have to put these connectors into a back box and put a front cover over it, which changes it to a much bigger job.

Inline Coupler from Kenable

So I decided to look for a better solution, and I found this Cat 6 Inline Coupler from Kenable.

It got 2 terminate blocks built-in, one for the incoming cable, and one for the outgoing. There is a diagram of the Type B protocol printed on the product, so I don’t have to look it up on my phone. All I have to do is punch down the 16 wires one by one. With a quality punch down tool it is a lot easier and quicker than I thought.


Inline Coupler In Use

The product itself is solid, much better quality than the LAP data module. I didn’t have to worry about damaging the terminal or face plate when pushing it against the wall while using a punch-down tool.

The size is on a sweet spot, about 24mm depth, just enough to tuck it into the 25mm service void. I am not sure if it is maintenance-free or not, but I am comfortable leaving it in the service void as it has an enclosing cover on it.

Kenable is the only place that sells it at a reasonable price, about £2 each, while the rest of the sellers is asking for £5 so thank you Kenable for making it affordable.


Full 500 Mbps Speed in the Office

Re-discovery the Ancient Info Documentation System in the Age of LLM

So there are few notes that helped me to learn Info. Hopefully it can
bring more new users to the Info system.

Table of Contents

  1. Travel with Info
  2. Why Info is not Popular?
  3. dir the Index File
  4. Setup Info in MacOS

Travel with Info

The best time to learn difficult thing is in travelling. During my last two-week’s trip to Singapore/Malaysian, I was reading about Ledger Cli causally. without putting much efforts, it clicked. It suddenly started to make sense to me.

The more I learn, the more I want to learn more. I cannot wait for the next opportunities to open Emacs and dive into Ledger’s brilliant documentation. This is me with my Emacs in Changi Airport next to the Jewe.


Emacsing next to the Rain Vortex in Changi Airport

I was able to apply the learning and came up with the project-rule to keep data hygiene (will blog next). The positive feedback energise me. The flight to London is ready for boarding but I don’t want stop exploring during the 14 hours flight without WIFI.

That’s where I re-discovered the Info documentation system. I used it to read the ledger.el library between sleep sessions 8000 feet above the ground. Reading in plain text inside of Emacs has great benefits, no distractions, fraction free in taking notes. it was a breeze.

Then I stepped into learning the Info documentation system itself, how to navigate, search text/index and all that. I was able to pick it up quickly, the concepts and shortcuts are native to me as an experienced Emacs user.

I envisioned myself to use it to read all the documentation, e.g. Pandas library’s in Python. That would be ideal I told myself. However, I soon realised that Info documentation system is a niche tool: it is mostly used in GNU projects and Emacs libraries.

Why it is no popular? I was wondering myself. I decided to have a go myself. well, the journey to start is already full of hiccups. This is typical theme in learning legacy system, and could put many people off.

So there are few notes that helped me to learn Info. Hopefully it can bring more new users to the Info system.

dir the Index File

The first and most important thing I realised is, in the context of Info, the dir is not a directory, but a plain text file. I simply call it index file, then the rest becomes so much clearer.

The =dir=/index file is the entry point of the Info program. it has a lists of the available Info manuals with their name, Info file location, and desecration.

Setup Info in MacOS

Then there is a bug in emacs-plus: during the installation of Emacs, the dir file somehow got deleted in the cleaning process. So the manuals for the default libraries that comes with Emacs are not available. In my case, I only have few Info from the packages I installed post-installation, like orderlies, org-roam for example.

I took a slightly different approach to fix this problem: I kept the system level tools separate from the Emacs’s library, so i have two dir files.

 
# manuals of system level programs
cd /opt/homebrew/share/info
for file in * ; do install-info "$file" dir; done

# manuals of Emacs and Emacs libraries
cd /opt/homebrew/share/emacs/info
for file in * ; do install-info "$file" dir; done

Then tell Emacs the locations of those dir files as below.

 
(setq Info-directory-list
      (list "/opt/homebrew/share/info"
            "/opt/homebrew/share/info/emacs"))

Note, the convention is for each directory in the list, there is a dir file, on in Emacs, we are specifying the file using directory, and the file is happened to be called dir. I feel the naming can be improved to avoid such confusion!

After restarting Emacs, Info will show there are about 500+ manuals available, e.g. find tool, mu4e library, and Ledger3.

Lastly, an quick note install-info. As shown above, it is used to install Info manuals, taking ledger3.info as an example, to install it requires

 
install-info ledger3.info /opt/homebrew/share/info/dir

After that, the following line is added to the /opt/homebrew/share/info/dir file.

 
* Ledger3: (ledger3).           Command-Line Accounting

a bit of explanation:

  • *: mark the starting of the entry
  • Ledger3: is the node/manual name
  • ledger3: inside of a parenthesis is the path to the Info file without extension.
  • Command-Line Accounting: is the description of the manual.

Filter Ledger Transactions using Tags

I have been testing using Ledger-Cli to track my expenses, so far I have found the tagging system useful. In my ledger journal, each transaction is associated with a project, for example, the below transaction is assigned to project “2024 Monitor Stand”

2024-12-08 Screwfix
    ; project: 2024 Monitor Stand
    Expenses:HomeImprovement:Tools            £ 4.99 
    Expenses:HomeImprovement:PPE             £ 19.98 
    Expenses:HomeImprovement:PPE             £ 14.99 
    ; :refund:
    Assets:Amex

This constraint I came up with helps avoid meaningless spending on new shiny tools. Operationally, imposing this limitation on my book provides flexible ways of querying the data.

For example, bring up the transactions that do not have projects assigned to:

 
ledger reg exp and "expr" "not has_meta('project')" \
       --format "| %(date) | %P | %(amount) | %(note) |\n"
Table 1: posts without project
Date Payee Amount Note
2024/12/22 Selco £ 30.570 ; CaberFloor p5 T&G 2400x600x18mm x 2

There is only one post that I forgot to add the project tag, so pretty good.

A bit of explanation of the ledger-cli query syntax

  • exp: check only accounts contain ‘exp’, in the ledger’s convention, it is all expending accounts, i.e. Expense::*
  • expr: invoke filters using expressions
  • has_meta(‘project’): check if the transactions have the metadata key ‘project’
  • and, not: logical operators
  • –format: specify the output formatting

Another use case is counting the number of transactions per project. I use the number of purchased items as a proxy to gauge the project size.

 
ledger reg exp and "expr" "has_meta('project')" \
       --format "%(meta('project'))\n"  \
       | sort |  uniq -c | sort -bgr
Table 2: Number of items purchased for each project
No. Items Project
39 2024 Loft Lights
34 2024 Loft Insulation
32 2025 Garage Conversion
8 2024 Monitor Stand
2 General

The data shows the “2024 Loft Lights” project is by far the largest . That was a simple project by itself, however, since that was my first electrical project, I had to purchase a lot of stuff, 1.5mm cables, clamps, grommets, connectors, switches, sockets etc.

Finally, I have the “refund” tag so I can flag up the items to remind of myself to check if I received the refund fully.

 
ledger reg "expr" "has_tag('refund')" \
        --format "| %(date) | %P | %(amount) | %(note) |\n"
Date Payee Amount Note
2024/12/08 Screwfix £ 14.990 Site Optimus Gel Knee Pads

So far I enjoyed the plain text accounting using ledger-cli. The format and syntax are simple, and yet I can do complicated queries.

Setup ssh-agent Systemd Service for Emacs

Problem Statement

My personal desktop is not booting (the motherboard is probably dead) so I have been setting my server so I can work while sorting things out.

I got stuck in getting magit working in emacsclient: I thought I could run ssh-add inside of Emacs that would allow magic to access my git repos using ssh, but apparently, it is not the case.

After some digging, I learnt that the problem I have to solve is to run one ssh-agent in the background and then make the Emacs/Magit or any programs hook onto it. Then once I run ssh-add and type the passphrase for the first time, either inside of Emacs or in a bash terminal, everything would work.

Implementation

Drop the following unit file below to ~/.config/systemd/user/ssh-agent.service.

[Unit]
Description=SSH key agent

[Service]
Type=simple
Environment=SSH_AUTH_SOCK=%t/ssh-agent.socket
ExecStart=/usr/bin/ssh-agent -D -a $SSH_AUTH_SOCK

[Install]
WantedBy=default.target

The important things are

  1. The environment variable SSH_AUTH_SOCK is specified. It can be anywhere as long as this environment variable in other programs points to the same location.
  2. ssh-agent is invoked with the -a option to provide an address specified in the above step.

The $t is a specifier1 in systemd, it is equivalent to $XDG_RUNTIME_DIR variable in Debian. It points to the runtime temporary directory which apparently is safer2 than the /tmp directory. The runtime directory was cleaned up after stopping the ssh-agent so it is non-persistent.

To start the ssh-agent service:

 
systemctl enable --user ssh-agent
systemctl start --user ssh-agent

After that, update the unit file of Emacs to include this line (follow up my blog post Managing Emacs Server as Systemd Service for the full setup).

Environment=SSH_AUTH_SOCK=%t/ssh-agent.socket

To make it work for bash shell and all other programs calling from a bash terminal, add this line to ~/.bashrc.

 
export SSH_AUTH_SOCK="$XDG_RUNTIME_DIR/ssh-agent.socket"

Alternatives

There are programs developed to solve this specific problem (see Debian wiki). While using such a program seems like a simpler alternative (e.g. keychain), I prefer to use systemd as the unified approach for managing background services. I have been using it for emacsclient, and I’m adding ssh-agent to it.

What is your preference? How do you solve this problem?

Footnotes

1 All the specifiers are listed here.

2 I am not a security expert but the StackExchange comments seem to make sense.

If you have any questions or comments, please post them below. If you liked this post, you can share it with your followers or follow me on Twitter!