On April, 4th, I received a promotional offer from Rogers offering Ignite Internet service at 1.5Gbps plus Streaming for $114.99 per month.
I procrastinated a bit because I wanted to make sure that I can actually make use of this service. However, when I checked my bill for April, I noticed that my total monthly charges is at $102.99.
Note the above price prior to discount is at $117.99. I was curious to see if Rogers can get me a good deal without the Streaming service. I called Rogers support line and received a person who was not very helpful and simply just quoted conditions and deals to me. AI will do a number of these types of people soon.
I decided to try an alternative route by using Twitter (@RogersHelp). I direct messaged Rogers on Twitter and received wonderful help. They offered me the 1.5Gbps service at only $104.99 (with a 24 months commitment). This will be somewhat on par with my current payment and I will get 50% more throughput.
There is another question. Will my networking equipment make use of the 1.5Gbps? My networking setup has the Rogers Ignite WiFi Gateway (ARRIS Group, XB7 Modem) and is connected with a Cat5e cable to my Unifi Dream Machine Pro, using one of its 1Gbps RJ45 port.
How can we overcome the 1Gbps limit on the UDM Pro’s RJ45 port? Luckily the UDM Pro has an 10G SFP+ port as well. I went to Amazon and purchased a 10G SFP+ RJ45 Copper Module Transceiver.
The above will auto negotiate a 2.5Gbps to 10Gbps connection from the XB7 to my UDM Pro. Of course I will not be getting 2.5G or 10G speeds. These are just the physical maximum per the respective devices. Rogers will throttle my inbound and outbound traffic to 1.5Gbps and 50Mbps respectively.
After installing the SFP+ module and rewire the existing Cat5e cable, I had to reboot the both the XB7 modem and the UDM Pro. Once everything came back up, I had another problem. How do I test that I actually get 1.5Gbps? I cannot do it from any WiFi devices or any wired devices in my house because they are all limited to the 1Gbps port speed from my networking switches. Once again, Unifi had thought of this already and provide a speed test functionality on its management dashboard.
The tested speed seems to be better than expected.
So you can see from the above screen shot, that we are now getting what we are paying for. I also performed a double test from two different machines that are routed to a switch that has a 10Gbps connection to my UDM Pro, and each machine received a 700Mbps to 800Mbps download speed, which is around 1.5Gbps in aggregate. Mission accomplished.
Unifi just came out with a new firmware update that enable the UDM Pro to perform load balancing of more than one WAN connection. When the SkyLink service becomes more economically feasible, we can attach a satellite based internet service as a compliment to the existing Rogers service. This way during a power outage, we can continue to get Internet.
Prior to the holidays in 2022, I upgraded my media NAS server as detailed here. After this upgrade, I repurposed the old server’s components and built another PC.
Originally I was going to use this extra PC as a simple online media encoder, since encoding videos in the HEVC codec takes a lot of CPU power. I did this for about a month. My son, Kalen had an old GTX1060 6GB graphics card that he was going to place on Kijiji for resale. I offered to purchase this graphics card off of him so that that I can pair it up with this repurposed PC. The new idea was to turn this PC into my gaming PC. I don’t do many 3D intensive gaming, so an older GPU is certainly good enough for me.
Off I went installing Windows 10 Pro on the PC. I also discovered at this time the Windows Subsystem for Linux (WSL). I thought it would be a wonderful idea for me to have the gaming PC and not lose the ability for the PC to double as a media encoder through the use of a Linux distribution using WSL. My hope is that Linux with WSL will yield near metal based performance. Long story short, the performance of ffmpeg, the tool that I use for video encoding, was disappointing. Apparently there is a bug in WSL v2 that forced ffmpeg to only use 50% of its CPU power. There was nothing wrong with the concept of having a dual purpose PC for gaming and a handy Linux distribution for other endeavours.
The problem is with the Windows hosted Hypervisor, a software layer that usually runs between the hardware and the operating. I know of another hypervisor called Proxmox. This is a perfect opportunity for me to try Proxmox out. Before I installed Proxmox, I maxed out the memory of this repurposed PC to 64GB. It only had 16GB before and I thought this would not be enough.
One of the worries I had was how to get the raw GPU performance from Proxmox? Apparently there is a GPU passthrough option. Before installing Proxmox, I had to make some BIOS adjustments on the PC.
Enable IOMMU
Enable SVM Mode (same as Intel VT-x)
Enable AMD ACS
Only the SVM Mode is required for Proxmox, the other two are required for GPU Passthrough. After I installed the Proxmox server, I followed the instructions outlined in the following sites:
The first site was more clear and was the most helpful. I used the second and third sites as an alternate source and backup reference. Thanks to the above sites, I was able to get Proxmox running and created two virtual machines (VM’s). The first is an Ubuntu distribution called workervm and the second is a Windows 10 Pro instance with a GPU passthrough, called win10. Below is a screenshot of the Proxmox control administration site.
Proxmox control panel (click to enlarge)
Below is the workervm (Linux VM) configuration:
workervm configuration for Ubuntu instance
I had to make sure the processor type is set to [host] to get the most performance out of the virtual CPU’s. The Windows VM configuration uses a different BIOS, specifically a UEFI BIOS. We also have to ensure that the Machine type is set to q35. The Windows VM also has the EFI Disk and TPM State configured, and of course the extra PCI Device to represent our GPU passthrough card. Check out the full configuration for the Windows 10 VM below:
win10 configuration for Windows 10 Pro instance
After installing Windows 10 Pro, the network interface is not recognized. To remedy this situation I had to install virtio-win as described by this site here. After the installation of virtio-win, and a reboot. I had networking, connectivity to the Internet, and the Device Manager output from the Windows 10 Pro instance as shown below. Notice that Windows recognized the native NVIDIA GeForce GTX 1060 6GB card.
Windows 10 Pro VM instance Device Manager
I tried to test out the GPU throughput with some 3D rendering demos and tested a couple of games from Steam using Remote Desktop. The performance was okay, but not stellar, and could have been better. I did some more research, and apparently Parsec, a virtual desktop sharing tool, is probably better for remote gaming.
I went ahead and installed Parsec on both the Windows 10 Pro VM, and on my Mac mini, which I used to remotely play games on the previous VM. This worked out to be quite well.
Now the repurpose PC is a Proxmox server hosting as many VM’s that the hardware can bear. The workervm instance can be used for video encoding and other generic Linux oriented work or trials. The win10 instance will be used for gaming and hosting our tax filing software, called TurboTax, which only runs on Windows.
In the near future, I will also be testing out Proxmox with virtual containers instead of machines. The containers are more light weight and less resource intensive. It will be another new adventure here.
Today is January 13, 2023. We had an icy snow storm last night that lasted until this morning, and I was curious what the roof condition was like. Just how much of the panels were covered in snow?
Solar energy for today
Our peak energy production was at around 11am when we generated a little over 800Wh, which is inlined with what we kind of get on a cloudy, misty, winter day. In contrast, the best we got so far was on January 7th at 1pm. We generated 5,494Wh. That was a sunny day with no snow coverage on the panels.
A quick drone survey of our roof this afternoon at around 3pm.
I was kind of impressed that we got that much with so much of the panels covered. Watch the above video to see just how much of the panels are covered today. Our total production for today is only about 3,400Wh.
Below are the stats per panel.
Per panel generation statistics for today.
As you can see above, every panel contributed even the covered ones! There will be two sunny days over the weekend, so we will see!
Update: 2023-01-14
I did another roof survey with my drone, seeing that today it was a sunny sky day.
Roof survey on Jan. 14 (day after storm)Solar energy production on Jan. 14
We have generated over 10,000 Wh of energy today about 3 times more than yesterday. The survey was conducted when it was still -6 ºC outside, so way below freezing.
I have a membership with Audible and I sometimes also get other audio book sources. Recently I experimented with combining all of my audio books into a centralized place. Since I already have a Plex server running, I thought it would be a good place to do this.
I did a little research and came across a couple of very helpful articles:
I have a single folder to store all of my audio books. Inside the folder, each audio book is stored as an “m4b” file.
Ensure that audio books have a poster image and that its artist and album_artist tags are set to the author. Where appropriate, the audio book should also contain chapter metadata.
Create a music library on Plex by adding the audio book folder, and set the agent to Audnexus
Note the Agent setting
Ensure that the advanced option of “Store track progress” is checked.
Ensure that Store Track Progress is checked!
Each book in the library will be represented as an album, and the author will be mapped to album artist. Once the library is created, you can download and play the audio books from the desktop using the Plex app. However, the more common use case is to listen to the audio books while on the go.
Using Prologue to Play Audio Books
We first have to download the Prologue App. I did not get any of the In-App premium functionality, and just stayed with the free version.
Point the app to my Plex server’s URL, and all the audio books from the library should now be accessible and playable on the iPhone or iPad with chapter, bookmark, and last-left-position support.
This is a really neat solution, and I am impressed how Plex and Prologue together formed a dynamic duo in this manner.
My wife and I were watching Lighter & Princess (点燃我，温暖你). In episode 5, a smart programmer completed a heart animation as part of a programming test in a university course. Everyone of his classmates had a mediocre and simplistic implementation.
My wife challenged me to do the same. I could have done this with SVG and Javascript, but I took the opportunity to learn more Python and Qt. The latter I have not used at all and was always curious how well it worked with Python. I decided to use the PyQt package instead of the PySide package.
I hacked this together today resulting in this rendition:
Click above to see it in action
The PyQt implementation worked really well across Windows, Mac, and Ubuntu Linux. I suppose the Javascript and SVG would have done the same, but I learned something new in this case.
The source code and the installation instructions are at:
In the first part of this post, I talked about making sure all the new hardware that I recently purchased works. Yesterday, upgrading from Ubuntu 20.04 LTS to 22.04 LTS was super simple. Unfortunately, that was the end of the easy part.
I thought I could just image by old boot drive and make a carbon copy of it on my new boot drive. My old boot drive is a simple SATA 512GB SSD, and my new boot drive is an NVMe M.2 1TB SSD plugged directly to the motherboard. The copying was pretty simple, but because the drives differ in size, I had to relayout the partition table with the new drive once the copy is completed. I did this with the parted command.
Unfortunately the new boot drive did not want to boot. At this point I had to do some research. The most helpful articles were:
Both of the above articles were an excellent refresher on how GRUB works. I have used GRUB since the beginning, but one gets super rusty when these types of tasks are only performed once every three or six years!
Instead of detailing what went wrong, I will just explain what I should have done. This way if I need it again in the future, it is here for my reference.
Step 1: Perform a backup of the old boot drive from a Live USB in shell mode. This is done on my server on a nightly basis. This method is clearly described on the Ubuntu Community Help Wiki.
Following this method I will end up with a compressed tar archive for my entire root directory, skipping some runtime and other unwanted directories.
Step 2: After installing a fresh install of the new Ubuntu LTS Server operating system on the new server and boot drive, I proceeded to backup the new boot with the same technique used in Step 1. I stored the backup of the new install on another external SSD drive that I have lying around. Also it is important that new boot drive partition layout of the new install contains a swap partition.
Step 3: I then restore the most recent backup (done in Step 1) of the old boot drive to the new boot drive. I then replaced the /boot/grub directory with the new contents from the new install which was backed up in Step 2. The new GRUB is already installed when we performed a brand new installation on the drive. We just want to make sure the boot partition matches the /boot/grub contents.
Step 4: We also need to fix up the /etc/fstab file because it contains references to drive devices from the old hardware. Paid special attention the main data partition and the swap partition. It should look something like this:
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
# / was on /dev/nvme1n1p2 during curtin installation
UUID=fc939be4-5292-4252-8120-7ef59b177e5b / ext4 defaults 0 1
# /boot/efi was on /dev/nvme0n1p1 during curtin installation
UUID=5187-A8C6 /boot/efi vfat defaults 0 1
# Swap partition
UUID=512d611e-6944-4a57-9748-ea68e9ec3fad none swap sw 0 0
# /dev/mapper/airvideovg2-airvideo /mnt/airvideo ext4 rw,noatime 0 0
UUID=9e78425c-c1f3-4285-9fa1-96cac9114c55 /mnt/airvideo ext4 rw,noatime 0 0
Noticed that I also added the LVM logical volume for /mnt/airvideo, which is my RAID-1 array. The UUID can be obtained by the blkid command. Below is a sample output:
Step 4B (Potentially): If the system boots in the “grub>” prompt, then we will have persuade grub to manually boot by providing the following at the prompt:
grub> set root=(hd9,gpt2)
grub> linux /boot/vmlinuz root=/dev/nvme1n1p2
grub> initrd /boot/initrd.img
grub> boot
To find the root value on the first line, you have use the ls command which is explained in this article. The root parameter on the linux line references the partition which the root directory is mounted. In my case, it was /dev/nvme1n1p2.
After I rebooted, I reinstalled GRUB with the following as super user:
grub-install /dev/nvme1n1
It may also be required to update our initramfs using:
update-initramfs -c -k all
Step 5: At this point the system should reboot and all of the old server’s content should now be on the old hardware. Unfortunately we will need to fix the network interface.
First obtain the MAC address of the network interface using:
% sudo lshw -C network | grep serial
serial: 04:42:1a:05:d3:c4
And then we will have to edit the /etc/netplan/00-installer-config.yaml file.
% cat /etc/netplan/00-installer-config.yaml
# This is the network config written by 'subiquity'
network:
ethernets:
enp6s0:
dhcp4: true
match:
macaddress: 04:42:1a:05:d3:c4
set-name: enp6s0
version: 2
Ensuring the MAC address matches from lshw and that the name is the same as the old system. The name in this example is enp6s0. We then need to execute the following commands to generate the interface.
netplan generate
netplan apply
We need to ensure the name matches because many services on the server have configurations that references the interface name, such as:
Configurations in /var/network/interfaces
Samba (SMB) (/etc/samba/smb.conf)
Pihole (/etc/pihole/setupVars.conf)
Homebridge (/var/lib/homebridge/config.json)
Step 6: Fix the router provisioning DHCP IP addresses so that the new server has the same fixed IP address as the old server. This is important because there may be firewall rules referencing this IP address directly. The hostname should have been automatically restored when we restored the partition in Step 3.
Step 7: Our final step is to test the various services and ensure they are working properly. These include:
On May 15th, 2019 (more than three years ago), I performed a performance boost to my media server by upgrading its CPU, Motherboard, and Memory. You can read that experience in this post.
Today, I am going to be doing the same. It looks like we are on a cadence of every 3 years or so to do a spec bump. This time around we are also changing the same items, but will include the power supply as well in the swap. I also decided to swap the boot drive hardware from an old SSD drive to an NVME drive. All of this resulted in the following hardware acquisitions, all from Amazon, which I find them to have lower pricing (when factoring free shipping through Prime), than Newegg even during Black Friday and Cyber Monday offers.
The plan is to spend the time today to roughly test out all the new hardware.
Test Setup
I quickly did a skeleton setup to make sure Ubuntu 22.04.01 Server Edition works with all hardware involved, especially the networking.
Memory Test
Once I know Ubuntu server is working good, I am now testing the server’s new 32GB DDR4 memory. This is running as I write this post and will let it run overnight.
The plan for tomorrow is to upgrade the current media server from Ubuntu 20.04.5 LTS to Ubuntu 22.04.1 LTS. Once this is done, I can then backup everything, and move the new hardware into the old casing and hope everything works.
This is going to be a fairly technical post on the topic of my Conext XWPro battery inverter configurations. I am writing this post primarily to document my experience and my current rationale, and for my future, forgetful self.
Previously I had my Grid Support SOC (State of Charge) and the Recharge SOC set to 40%. With these settings, the battery will be used (anytime during the day and night) until it discharged to 40%, which will initiate a charge cycle that will charge the battery back to 100%. Under normal circumstances, the battery will typically discharge during a very cloudy day, but mostly in the evenings and at night.
I had the above settings because I stupidly thought I should stay off the grid as much as possible. The intent is to try to charge the batteries in the evening during off peak hours, and not to use the grid at all during on peak hours. These settings certainly accomplish this, but at the expense of shortening battery life. Another big downside with this approach is that charging the battery through the Conext XWPro inverters only achieves around 83% efficiency. This observation is based on my real-time data observations from the actual inverters.
Yesterday I noticed that my batteries are reporting a State of Health (SOH) drop to 99% instead of 100%. This was a bit alarming given only 6 months of uses. I also realized from the Alectra invoices that Time of Use (ToU) is not a factor in Alectra’s billing calculations. All of this resulted in a shift in my thinking. We will now use the grid as our primary battery, and preserve our Lithium Ion as our backup batteries only. Time shifting of loads will no longer be my primary concern since it is no longer worth it with zero benefits.
To do this, I have set the Grid Support SOC to 90% and the Recharge SOC set to 85%. This way immediately after a charge cycle, the battery will be used a little bit to draw down from 100% to 90%. This has two benefits in my opinion. The first is to get some charge flow through the batteries, so it is not only sitting there. The second is that it leaves a 10% SOC gap. If we have a power failure during a sunny day, there is space for the excess solar production to go without tripping the solar inverters.
The 5% gap between Grid Support and Recharge is currently a guess. My thinking is that over time the charge on the Lithium batteries will leak and it will trigger a recharge cycle. Of course I did not want to set the Recharge the same as the Grid Support, because this will cause a constant recharge loop which defeats the purpose of preserving battery life. I do not know how long it will take to naturally draw down from 90% to 85%. This is why it is still a guess at this point. If there is no leakage, which is great news because it shows how good the batteries are, then I will have to trigger a recharge cycle at least once a month just to keep the charge flow within the battery’s chemistry.
For now I will live with the new settings and see how often the battery cycles. If it only cycles once every one or two months, then that is perfect. If it does not cycle through more than three months, then I may have to add the monthly charging cycle logic into my custom controller.
Our home Network Attached Storage (NAS) media server is going below 4 Terabytes of free space. The Seagate IronWolf 12TB hard drives were on sale with Amazon offering them below $300. I figure that I swap out two old 6TB drives with these new 12TB drives resulting in a net increase of a further 6TB of storage.
The last time this was done was around two years ago when I replaced 4TB and 6TB hard drives with 10TB hard drives.
So far the mdadm and LVM storage architecture has proven to be very flexible. I am able to mix drives of different sizes and able to grow our media storage volume over time.
Previously I had to make two swaps, each swap for each drive in the array. Effectively I am changing two 6TB drives for two 12TB drives because they are in a Raid 1 array. I cannot swap both at the same time, because I have to incrementally sync the data from the old drives to the new ones.
This has always been inconvenient because it means opening the physical server twice. However, this time I used my USB 3.0 HDD dock. I inserted one of two 12TB new drives into the dock, and then I temporarily created a three disks Raid 1 array. Once the sync is completed, which took 10+ hours, I remove one 6TB drive from the array configuration and I then physically replace both 6TB drives with both 12TB new drives in the server chassis, and place one old 6TB drive into the dock. The 6TB drive in the dock is the one that is still in the array configuration. I then add the second 12TB drive that is already in the server chassis to the three disk array. Once again, a sync is required to accommodate the second 12TB drive. This also took 10+ hours. Once the second sync is completed, I can finally remove the second 6TB drive in the dock from the array and have the array returned back to a two disk Raid 1 array.
The above description is probably quite confusing, but this technique allowed me to just have a single down time for the server instead of two when swapping hard drives in the server chassis.
There will be an additional downtime when I grow or resize the LVM volume and file system.
After this upgrade I should have the following Raid 1 (fully mirrored) arrays:
An array with 2 x 8TB
An array with 2 x 10TB
An array with 2 x 10TB
An array with 2 x 12TB
The above four arrays are combined into a logical volume using LVM that results in a total volume size of 40TB (fully mirrored) or a little over 36TiB of usable space (increasing from the old 31TiB).
We now have been running our net smart meter for more than a day now. I mentioned that we got our new net meter on this previous post.
Of course I am now curious how to read the meter so that I can decipher how much electricity we sent back to the grid. Here is a short video of what the meter is showing:
Meter Display Sequence
Initially the displayed information is quite cryptic, but looking at the meter’s label, I found this group of small prints.
These labels essentially tells us what is going on. The LED display cycles through 5 modes in total. The initial display is a segment test, which means all segments of the LED are displayed. This is a simple test to ensure that the LED display itself is functioning correctly. Next, it shows LST003, indicating that the next number it shows will be the amount of kWh of electricity that we ended up consuming or using. This is followed by LST004, another label indicating that the following number is the amount of kWh of electricity that is sent back to the grid.
Now with this new found knowledge, the above video shows that we used 13 kWh and exported 103 kWh since the meter was installed in the afternoon of May 4th.
In about 1.5 days, and bright sunny day yesterday, we generated and provided to our community electricity grid with a net of 90 kWh of energy.
Excerpt from the CBC article from Oct. 8th, 2021
There are 4 people in our house right now, and according to a recent CBC article our average carbon footprint is about 14.2 tonnes of CO2 per person. Doing a little more research, I found this white paper titled, “A Clearer View on Ontarios Emissions June 2019“. On page 8 of this paper, we see an annual average emissions factor (AEF) of 31 grams of CO2 per kWh. One tonne is 1,000,000 (a million) grams. This means to offset one individual, we need to offset 14,200,000 grams of CO2, and using the AEF this is equivalent to approximately 458,065 kWh!
To put this big number in perspective, I think our last month’s electricity bill only shows us using around 1,200 kWh of electricity.
It is clear that we will not be able to offset one of us, never mind all four of us by just using solar ourselves (at least not in Ontario). The idea of carbon neutrality is still a long ways off, and the above numbers show that we cannot do it alone. It will require every industry to do its part.
Update 2022-05-26 2:45pm: Took another reading outside. Used 313 kWh, Exported 1018 kWh, a net of 705 kWh. This with about 23 days of operation since May 04th.
