Playing with Proxmox

Prior to the holidays in 2022, I upgraded my media NAS server as detailed here. After this upgrade, I repurposed the old server’s components and built another PC.

Originally I was going to use this extra PC as a simple online media encoder, since encoding videos in the HEVC codec takes a lot of CPU power. I did this for about a month. My son, Kalen had an old GTX1060 6GB graphics card that he was going to place on Kijiji for resale. I offered to purchase this graphics card off of him so that that I can pair it up with this repurposed PC. The new idea was to turn this PC into my gaming PC. I don’t do many 3D intensive gaming, so an older GPU is certainly good enough for me.

Off I went installing Windows 10 Pro on the PC. I also discovered at this time the Windows Subsystem for Linux (WSL). I thought it would be a wonderful idea for me to have the gaming PC and not lose the ability for the PC to double as a media encoder through the use of a Linux distribution using WSL. My hope is that Linux with WSL will yield near metal based performance. Long story short, the performance of ffmpeg, the tool that I use for video encoding, was disappointing. Apparently there is a bug in WSL v2 that forced ffmpeg to only use 50% of its CPU power. There was nothing wrong with the concept of having a dual purpose PC for gaming and a handy Linux distribution for other endeavours.

The problem is with the Windows hosted Hypervisor, a software layer that usually runs between the hardware and the operating. I know of another hypervisor called Proxmox. This is a perfect opportunity for me to try Proxmox out. Before I installed Proxmox, I maxed out the memory of this repurposed PC to 64GB. It only had 16GB before and I thought this would not be enough.

One of the worries I had was how to get the raw GPU performance from Proxmox? Apparently there is a GPU passthrough option. Before installing Proxmox, I had to make some BIOS adjustments on the PC.

  • Enable IOMMU
  • Enable SVM Mode (same as Intel VT-x)
  • Enable AMD ACS

Only the SVM Mode is required for Proxmox, the other two are required for GPU Passthrough. After I installed the Proxmox server, I followed the instructions outlined in the following sites:

  1. From 3os.org: GPU Passthrough to VM;
  2. From pve.proxmox.com;
  3. And from reddit.

The first site was more clear and was the most helpful. I used the second and third sites as an alternate source and backup reference. Thanks to the above sites, I was able to get Proxmox running and created two virtual machines (VM’s). The first is an Ubuntu distribution called workervm and the second is a Windows 10 Pro instance with a GPU passthrough, called win10. Below is a screenshot of the Proxmox control administration site.

Proxmox control panel (click to enlarge)

Below is the workervm (Linux VM) configuration:

workervm configuration for Ubuntu instance

I had to make sure the processor type is set to [host] to get the most performance out of the virtual CPU’s. The Windows VM configuration uses a different BIOS, specifically a UEFI BIOS. We also have to ensure that the Machine type is set to q35. The Windows VM also has the EFI Disk and TPM State configured, and of course the extra PCI Device to represent our GPU passthrough card. Check out the full configuration for the Windows 10 VM below:

win10 configuration for Windows 10 Pro instance

After installing Windows 10 Pro, the network interface is not recognized. To remedy this situation I had to install virtio-win as described by this site here. After the installation of virtio-win, and a reboot. I had networking, connectivity to the Internet, and the Device Manager output from the Windows 10 Pro instance as shown below. Notice that Windows recognized the native NVIDIA GeForce GTX 1060 6GB card.

Windows 10 Pro VM instance Device Manager

I tried to test out the GPU throughput with some 3D rendering demos and tested a couple of games from Steam using Remote Desktop. The performance was okay, but not stellar, and could have been better. I did some more research, and apparently Parsec, a virtual desktop sharing tool, is probably better for remote gaming.

I went ahead and installed Parsec on both the Windows 10 Pro VM, and on my Mac mini, which I used to remotely play games on the previous VM. This worked out to be quite well.

Now the repurpose PC is a Proxmox server hosting as many VM’s that the hardware can bear. The workervm instance can be used for video encoding and other generic Linux oriented work or trials. The win10 instance will be used for gaming and hosting our tax filing software, called TurboTax, which only runs on Windows.

In the near future, I will also be testing out Proxmox with virtual containers instead of machines. The containers are more light weight and less resource intensive. It will be another new adventure here.

Panel Snow Coverage

Today is January 13, 2023. We had an icy snow storm last night that lasted until this morning, and I was curious what the roof condition was like. Just how much of the panels were covered in snow?

Solar energy for today

Our peak energy production was at around 11am when we generated a little over 800Wh, which is inlined with what we kind of get on a cloudy, misty, winter day. In contrast, the best we got so far was on January 7th at 1pm. We generated 5,494Wh. That was a sunny day with no snow coverage on the panels.

A quick drone survey of our roof this afternoon at around 3pm.

I was kind of impressed that we got that much with so much of the panels covered. Watch the above video to see just how much of the panels are covered today. Our total production for today is only about 3,400Wh.

Below are the stats per panel.

Per panel generation statistics for today.

As you can see above, every panel contributed even the covered ones! There will be two sunny days over the weekend, so we will see!

Update: 2023-01-14

I did another roof survey with my drone, seeing that today it was a sunny sky day.

Roof survey on Jan. 14 (day after storm)
Solar energy production on Jan. 14

We have generated over 10,000 Wh of energy today about 3 times more than yesterday. The survey was conducted when it was still -6 ºC outside, so way below freezing.

Media Server Upgrade 2022 (Part 2)

Part 1

In the first part of this post, I talked about making sure all the new hardware that I recently purchased works. Yesterday, upgrading from Ubuntu 20.04 LTS to 22.04 LTS was super simple. Unfortunately, that was the end of the easy part.

I thought I could just image by old boot drive and make a carbon copy of it on my new boot drive. My old boot drive is a simple SATA 512GB SSD, and my new boot drive is an NVMe M.2 1TB SSD plugged directly to the motherboard. The copying was pretty simple, but because the drives differ in size, I had to relayout the partition table with the new drive once the copy is completed. I did this with the parted command.

Unfortunately the new boot drive did not want to boot. At this point I had to do some research. The most helpful articles were:

Both of the above articles were an excellent refresher on how GRUB works. I have used GRUB since the beginning, but one gets super rusty when these types of tasks are only performed once every three or six years!

Instead of detailing what went wrong, I will just explain what I should have done. This way if I need it again in the future, it is here for my reference.

Step 1: Perform a backup of the old boot drive from a Live USB in shell mode. This is done on my server on a nightly basis. This method is clearly described on the Ubuntu Community Help Wiki.

Following this method I will end up with a compressed tar archive for my entire root directory, skipping some runtime and other unwanted directories.

Step 2: After installing a fresh install of the new Ubuntu LTS Server operating system on the new server and boot drive, I proceeded to backup the new boot with the same technique used in Step 1. I stored the backup of the new install on another external SSD drive that I have lying around. Also it is important that new boot drive partition layout of the new install contains a swap partition.

Step 3: I then restore the most recent backup (done in Step 1) of the old boot drive to the new boot drive. I then replaced the /boot/grub directory with the new contents from the new install which was backed up in Step 2. The new GRUB is already installed when we performed a brand new installation on the drive. We just want to make sure the boot partition matches the /boot/grub contents.

Step 4: We also need to fix up the /etc/fstab file because it contains references to drive devices from the old hardware. Paid special attention the main data partition and the swap partition. It should look something like this:

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/nvme1n1p2 during curtin installation
UUID=fc939be4-5292-4252-8120-7ef59b177e5b / ext4 defaults 0 1

# /boot/efi was on /dev/nvme0n1p1 during curtin installation
UUID=5187-A8C6 /boot/efi vfat defaults 0 1

# Swap partition
UUID=512d611e-6944-4a57-9748-ea68e9ec3fad	none	swap	sw	0	0

# /dev/mapper/airvideovg2-airvideo /mnt/airvideo ext4 rw,noatime 0 0
UUID=9e78425c-c1f3-4285-9fa1-96cac9114c55 /mnt/airvideo ext4 rw,noatime 0 0

Noticed that I also added the LVM logical volume for /mnt/airvideo, which is my RAID-1 array. The UUID can be obtained by the blkid command. Below is a sample output:

% blkid
/dev/sdf1: UUID="60024298-9915-3ad8-ae6c-ed7adc98ee62" UUID_SUB="fe08d23c-8e11-e02b-63f9-1bb806046db7" LABEL="avs:4" TYPE="linux_raid_member" PARTLABEL="primary" PARTUUID="552bdff7-182f-40f0-a378-844fdb549f07"
/dev/nvme0n1p1: UUID="r2rLMD-BEnc-wcza-yvro-chkB-1vB6-6Jtzgz" TYPE="LVM2_member" PARTLABEL="primary" PARTUUID="6c85af69-19a0-4720-9588-808bc0d818f7"
/dev/sdd1: UUID="34c6a19f-98ea-0188-bb3f-a5f5c3be238d" UUID_SUB="4174d106-cae4-d934-3ed4-5057531acb3c" LABEL="avs:3" TYPE="linux_raid_member" PARTLABEL="primary" PARTUUID="2fc4e9ad-be4b-48aa-8115-f32472e61005"
/dev/sdb1: UUID="ac438ac6-344a-656b-387f-017036b0fafa" UUID_SUB="0924dc67-cd3f-dec5-1814-ab46ebdf2fbe" LABEL="avs:1" TYPE="linux_raid_member" PARTUUID="29e7cfce-9e7b-4067-a0ca-453b39e0bd3d"
/dev/md4: UUID="gjbtdL-homY-wyRG-rUBw-lFgm-t0vZ-Gi8gSz" TYPE="LVM2_member"
/dev/md2: UUID="0Nky5e-52t6-b1uZ-GAIl-4Ior-XWTz-wFpHh1" TYPE="LVM2_member"
/dev/sdi1: UUID="5b483ac2-5b7f-4951-84b2-08adc602f705" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="data" PARTUUID="e0515517-9fbb-4d8a-88ad-674622f20e00"
/dev/sdg1: UUID="3d1afb64-8785-74e6-f9be-b68600eebdd5" UUID_SUB="c146cd05-8ee8-5804-b921-6d87cdd4a092" LABEL="avs:2" TYPE="linux_raid_member" PARTLABEL="lvm" PARTUUID="2f25ec17-83c4-4c0b-8653-600283d58109"
/dev/sde1: UUID="34c6a19f-98ea-0188-bb3f-a5f5c3be238d" UUID_SUB="8aabfe5b-af16-6e07-17c2-3f3ceb1514e3" LABEL="avs:3" TYPE="linux_raid_member" PARTLABEL="primary" PARTUUID="2fc4e9ad-be4b-48aa-8115-f32472e61005"
/dev/sdc1: UUID="ac438ac6-344a-656b-387f-017036b0fafa" UUID_SUB="c188f680-01a8-d5b2-f8bc-9f1cc1fc3598" LABEL="avs:1" TYPE="linux_raid_member" PARTUUID="29e7cfce-9e7b-4067-a0ca-453b39e0bd3d"
/dev/nvme1n1p2: UUID="fc939be4-5292-4252-8120-7ef59b177e5b" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="912e805d-fe68-48f8-b845-9bba0e3e8c78"
/dev/nvme1n1p3: UUID="512d611e-6944-4a57-9748-ea68e9ec3fad" TYPE="swap" PARTLABEL="swap" PARTUUID="04ac46ff-74f3-499a-814d-32082f6596d2"
/dev/nvme1n1p1: UUID="5187-A8C6" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="fe91a6b2-9cd3-46af-813a-b053a181af52"
/dev/sda1: UUID="3d1afb64-8785-74e6-f9be-b68600eebdd5" UUID_SUB="87fe80a1-4a79-67f3-273e-949e577dd5ee" LABEL="avs:2" TYPE="linux_raid_member" PARTUUID="c8dce45e-5134-4957-aee9-769fa9d11d1f"
/dev/md3: UUID="XEJI0m-PEmZ-VFiI-o4h0-bnQc-Y3Be-3QHB9n" TYPE="LVM2_member"
/dev/md1: UUID="usz0sA-yO01-tlPL-12j2-2C5r-Ukhc-9RLCaX" TYPE="LVM2_member"
/dev/mapper/airvideovg2-airvideo: UUID="9e78425c-c1f3-4285-9fa1-96cac9114c55" BLOCK_SIZE="4096" TYPE="ext4"
/dev/sdh1: UUID="60024298-9915-3ad8-ae6c-ed7adc98ee62" UUID_SUB="a1291844-6587-78b0-fcd1-65bc367068e5" LABEL="avs:4" TYPE="linux_raid_member" PARTLABEL="primary" PARTUUID="ed0274b9-21dc-49bf-bdda-566b2727ddc2"

Step 4B (Potentially): If the system boots in the “grub>” prompt, then we will have persuade grub to manually boot by providing the following at the prompt:

grub> set root=(hd9,gpt2)
grub> linux /boot/vmlinuz root=/dev/nvme1n1p2
grub> initrd /boot/initrd.img
grub> boot

To find the root value on the first line, you have use the ls command which is explained in this article. The root parameter on the linux line references the partition which the root directory is mounted. In my case, it was /dev/nvme1n1p2.

After I rebooted, I reinstalled GRUB with the following as super user:

grub-install /dev/nvme1n1

It may also be required to update our initramfs using:

update-initramfs -c -k all

Step 5: At this point the system should reboot and all of the old server’s content should now be on the old hardware. Unfortunately we will need to fix the network interface.

First obtain the MAC address of the network interface using:

% sudo lshw -C network | grep serial   
    serial: 04:42:1a:05:d3:c4

And then we will have to edit the /etc/netplan/00-installer-config.yaml file.

% cat /etc/netplan/00-installer-config.yaml 
# This is the network config written by 'subiquity'
network:
  ethernets:
    enp6s0:
      dhcp4: true
      match:
        macaddress: 04:42:1a:05:d3:c4
      set-name: enp6s0
  version: 2

Ensuring the MAC address matches from lshw and that the name is the same as the old system. The name in this example is enp6s0. We then need to execute the following commands to generate the interface.

netplan generate
netplan apply

We need to ensure the name matches because many services on the server have configurations that references the interface name, such as:

  • Configurations in /var/network/interfaces
  • Samba (SMB) (/etc/samba/smb.conf)
  • Pihole (/etc/pihole/setupVars.conf)
  • Homebridge (/var/lib/homebridge/config.json)

Step 6: Fix the router provisioning DHCP IP addresses so that the new server has the same fixed IP address as the old server. This is important because there may be firewall rules referencing this IP address directly. The hostname should have been automatically restored when we restored the partition in Step 3.

Step 7: Our final step is to test the various services and ensure they are working properly. These include:

  • Mail
  • Our web site lufamily.ca
  • Homebridge
  • Plex
  • Pihole (DNS server)
  • SMB (File sharing)

Finally the new system is completed!

New system all up and running!

Another Car Reservation

Last year on July 27, 2021, I placed a reservation for a RAV4 Prime with a Toyota dealer. It is now over one year, and the latest news from the dealer is that I am in position number three. However, they are currently only getting one or two cars a year!

Our initial take on the RAV4 Prime is that it is a hybrid, so it eliminates any range anxiety while still satisfies any day-to-day trips with a 60km all battery range. We have good experience with our Prius Prime which offers us a similar hybrid experience but with only a 35-40 km battery range.

Toyota RAV 4 Prime

While we continue the wait, it looks like many more electric vehicles (other than Tesla’s) are coming on the scene. There are recent additions from BYD, Polestar, Ford, Hyundai, and KIA. What caught my eye from a recent YouTube-surfing-session is the Hyundai Ioniq 5. This vehicle was introduced last December and is now “available for sale” in Canada. It has comparable range (~400km) and charge speed (350 kW DC) with the Tesla Model Y, sans the hefty price tag.

KIA EV6

The styling and look of the Ioniq 5 was not appealing to my taste. I then learned that the KIA EV6 is essentially the same vehicle but has a more traditional and sporty styling. Also a quick online build & price investigation showed that the Ioniq is a couple of thousand more expensive if we want to match the AWD long range trims.

So after much YouTube and online research, today I placed another car reservation for the KIA EV6. I opted for the trim named, AWD Long Range with GT-Line Package 1. I skipped the sunroof and the more fancied seats.

The bad news is that the sales guy is projecting a “three years” wait! He says that much of this will depend on supply chain issues. There is a good chance that it will be much sooner than the current projection.

On a side note, here is something else I discovered relating to KIA quality.

JD Power: Click for orignal source

I did not realize KIA ranked so high. The Buick and Dodge brand frankly surprised me as well. I wonder the accuracy on the above report, so take it for what it’s worth.

Nevertheless, I am keeping the RAV 4 reservation to see what options I have in 2023. Today, I also discovered that the Model Y may get a price cut and start sporting the new LFP batteries from CATL.

We will see! Who knew that buying an EV in 2022 is so difficult! This does not bold well for the planet.

Another NAS Storage Upgrade

Our home Network Attached Storage (NAS) media server is going below 4 Terabytes of free space. The Seagate IronWolf 12TB hard drives were on sale with Amazon offering them below $300. I figure that I swap out two old 6TB drives with these new 12TB drives resulting in a net increase of a further 6TB of storage.

The last time this was done was around two years ago when I replaced 4TB and 6TB hard drives with 10TB hard drives.

So far the mdadm and LVM storage architecture has proven to be very flexible. I am able to mix drives of different sizes and able to grow our media storage volume over time.

Previously I had to make two swaps, each swap for each drive in the array. Effectively I am changing two 6TB drives for two 12TB drives because they are in a Raid 1 array. I cannot swap both at the same time, because I have to incrementally sync the data from the old drives to the new ones.

This has always been inconvenient because it means opening the physical server twice. However, this time I used my USB 3.0 HDD dock. I inserted one of two 12TB new drives into the dock, and then I temporarily created a three disks Raid 1 array. Once the sync is completed, which took 10+ hours, I remove one 6TB drive from the array configuration and I then physically replace both 6TB drives with both 12TB new drives in the server chassis, and place one old 6TB drive into the dock. The 6TB drive in the dock is the one that is still in the array configuration. I then add the second 12TB drive that is already in the server chassis to the three disk array. Once again, a sync is required to accommodate the second 12TB drive. This also took 10+ hours. Once the second sync is completed, I can finally remove the second 6TB drive in the dock from the array and have the array returned back to a two disk Raid 1 array.

The above description is probably quite confusing, but this technique allowed me to just have a single down time for the server instead of two when swapping hard drives in the server chassis.

There will be an additional downtime when I grow or resize the LVM volume and file system.

After this upgrade I should have the following Raid 1 (fully mirrored) arrays:

  • An array with 2 x 8TB
  • An array with 2 x 10TB
  • An array with 2 x 10TB
  • An array with 2 x 12TB

The above four arrays are combined into a logical volume using LVM that results in a total volume size of 40TB (fully mirrored) or a little over 36TiB of usable space (increasing from the old 31TiB).

% df -h
Filesystem                        Size  Used Avail Use% Mounted on
udev                              7.7G     0  7.7G   0% /dev
tmpfs                             1.6G  3.4M  1.6G   1% /run
/dev/sdj1                         454G   64G  367G  15% /
tmpfs                             7.7G   37M  7.7G   1% /dev/shm
tmpfs                             5.0M  4.0K  5.0M   1% /run/lock
tmpfs                             7.7G     0  7.7G   0% /sys/fs/cgroup
/dev/mapper/airvideovg2-airvideo   37T   26T  9.1T  74% /mnt/airvideo
tmpfs                             1.6G     0  1.6G   0% /run/user/997
tmpfs                             1.6G     0  1.6G   0% /run/user/1000
/dev/sda1                         5.5T  548G  4.7T  11% /mnt/6tb

As you can see from above, the /mnt/airvideo now has 9.1TiB free!

The NAS motherboard and CPU is now over three years old. I may give it a couple of more years before considering another hardware upgrade.

Reading Our Net Smart Meter, Carbon Neutrality?

We now have been running our net smart meter for more than a day now. I mentioned that we got our new net meter on this previous post.

Of course I am now curious how to read the meter so that I can decipher how much electricity we sent back to the grid. Here is a short video of what the meter is showing:

Meter Display Sequence

Initially the displayed information is quite cryptic, but looking at the meter’s label, I found this group of small prints.

These labels essentially tells us what is going on. The LED display cycles through 5 modes in total. The initial display is a segment test, which means all segments of the LED are displayed. This is a simple test to ensure that the LED display itself is functioning correctly. Next, it shows LST003, indicating that the next number it shows will be the amount of kWh of electricity that we ended up consuming or using. This is followed by LST004, another label indicating that the following number is the amount of kWh of electricity that is sent back to the grid.

Now with this new found knowledge, the above video shows that we used 13 kWh and exported 103 kWh since the meter was installed in the afternoon of May 4th.

In about 1.5 days, and bright sunny day yesterday, we generated and provided to our community electricity grid with a net of 90 kWh of energy.

Excerpt from the CBC article from Oct. 8th, 2021

There are 4 people in our house right now, and according to a recent CBC article our average carbon footprint is about 14.2 tonnes of CO2 per person. Doing a little more research, I found this white paper titled, “A Clearer View on Ontarios Emissions June 2019“. On page 8 of this paper, we see an annual average emissions factor (AEF) of 31 grams of CO2 per kWh. One tonne is 1,000,000 (a million) grams. This means to offset one individual, we need to offset 14,200,000 grams of CO2, and using the AEF this is equivalent to approximately 458,065 kWh!

To put this big number in perspective, I think our last month’s electricity bill only shows us using around 1,200 kWh of electricity.

It is clear that we will not be able to offset one of us, never mind all four of us by just using solar ourselves (at least not in Ontario). The idea of carbon neutrality is still a long ways off, and the above numbers show that we cannot do it alone. It will require every industry to do its part.

Update 2022-05-26 2:45pm: Took another reading outside. Used 313 kWh, Exported 1018 kWh, a net of 705 kWh. This with about 23 days of operation since May 04th.

Net Meter Installed

Today is a good day. Alectra finally installed the net meter. From my previous post, I noted that without a net meter, any excess energy being sent back to the grid will be interpreted as usage. With the addition of the net meter, we can finally export our excess electricity from our solar panels without being charged for the generation. Instead, we can start earning and storing credits for the excess energy that we will supply to the grid.

Prior to the presence of the net meter, we gained plenty of experiences on going off grid. Effectively even on a cloudy day, we were able to generate enough energy for the house and charge our batteries to get us through the night. Below is a depiction of our energy utilization from Alectra.

Started to go off grid on April 23rd

The process of getting this net meter installed was not an easy feat! It took 22 days from the time of ESA inspection (April 12th) to Alectra installing the net meter. In summary, we played with the solar system to see what it can do for 11 days (April 12th to 22nd), while paying for the excess generation, and went off grid for the remaining 12 days (April 23rd to May 4th).

The small usages from the 23rd to the 30th that you see above were primarily charging our Toyota Prius Prime from the garage. That circuit is still grid tied and is independent of our Solar system. I cannot get an updated chart that contains data all the way up to today. Perhaps Alectra is doing something in the background in preparation for them to switch to net metering. In summary, we were pretty much off grid from April the 23rd to around 2pm today (May 4th). There was one exception, when we charged our backup batteries during off-peak hours from the grid on the evening of the 26th. We didn’t have to, but I was bit anxious with the battery at 50% whether it will last through the night and to the next evening, so this was more of an insurance. As we get more experience, we now have the confidence that even during cloudy / rainy days in the month of April, we should have no problem charging the batteries from solar that will last to the next night time operation.

Our old unidirectional meter
Our new bidirectional net meter

Once again, I have to thank New Dawn Energy Solutions for their correspondence and baby sitting the net meter installation process, as well as closing out the building permit from Richmond Hill. I am certain there was plenty of red tape that must be cut by them to get to where I am today, so kudos to them!

Today was also the day when we completed our second audit with Enertest. Once again Nick Crosby, A Certified Energy Advisor did a professional job. This audit is mandatory for the participation of the Canada Greener Homes Grant program.

If you are thinking of installing solar, New Dawn and Enertest are partners and experts in your endeavours.

Solar Power and Off Grid Operations

In my previous post, all major installations were completed. Since that time, the ESA inspection was completed and we validated our batteries so that we have confidence that they will last for more than a day in the worst case scenario (no sun). However at the time of this writing, we are still waiting for Alectra Utilities to switch out our old meter to a new one that is net-meter capable. Until this meter replacement occurs, every watt-hour (Wh) of energy we produce and send back to the grid, Alectra will charge us for it as if we are using that energy instead of producing it. Here is a summary of the timeline from panel installation:

  • Solar panels installation completed on April 8th;
  • ESA Inspection on April 12th;
  • New LiFePO4 batteries installed on April 18th;
  • From April 18th onwards, we tested the system through a series of scenarios;
Our Utilization Chart from Alectra Utilities (click to enlarge)

So prior to the ESA inspection on April 12th, we continue our on-peak time shifting. You can see that there has been very little on-peak usage (red indicator) before April 12th. Once the ESA inspection is completed, we turned on our solar panels for the first time.

The erratic “usage” indicated in the above chart after April 12th, is a direct result of excess solar energy being exported back to the grid. Since our net meter has yet to be installed, Alectra sees it as usage, and unfortunately I will have to pay for that generation, very ironic if you ask me.

Nevertheless, we gathered much data in the last couple of weeks. We tested the system for both on grid and off grid operations. We tested with washer and dryer loads. Today on a bright sunny day, I even tried our air conditioner when we are off grid. The air conditioner started without any issues and worked with just solar energy, impressive. I will try again at night when we only use the batteries.

Let us take a look at our energy generation data that we collected so far. The information here is a surprise to us in a good way. The best way to show this is to provide the data for our best day performance to date.

Our best performance day (April, 20th), 103.63 kWh generated

On April 20th we had a beautiful sunny day. We generated 103.63 kWh of electricity, since the house could not use it all, we fed most of it back to the grid. This is an excellent run and really show what the panels are capable of. For comparison, our average daily use is between 30 to 40 kWh. This means our solar generation ability on a sunny day can easily cover 2.5 to 3 days. For those Tesla drivers out there, we can generate enough power to fill your “tank”.

A rainy and cloudy morning and the sun came out at around 3pm.
Total generation: 32.53 kWh (not whole day)

Yesterday was a rainy and cloudy morning, and the power generation on average kept up with house load usage. We woke up with the batteries at about 50% charged and the system managed to gain around 10% of battery charge at 3pm. After 3pm, the sun started to come out and the batteries charged rapidly. It easily reached 87% state of charge, and I had to shut the solar generation down at around 5pm, otherwise the energy would have no place to go, which leads to another major dilemma for off grid operation.

During on grid operation, the grid can regulate and absorb the excess energy generated by our solar panels. This is a huge convenience, which until we have the net meter, we really cannot take advantage of.

During off grid operation, we must use all the energy generated. Our supply must match demand and vice versa. This is where the batteries come in. They help to buffer or store the excess, and supplement any shortages. However, when the batteries are full and our usage cannot keep up with the generation, then the best option is to shutdown the solar, and shift our energy consumption to the batteries. Using the batteries will create more “empty” capacity, which we can later use to store more sun energy. I assumed, incorrectly, that this power regulation will be handled by the Schneider inverters. This is not the case, at least not fully. I am not going to go into details of Frequency Shift Power Control and other inverter deficiencies here, but suffice it to say that they are really not that smart. We will have to investigate on a more flexible power regulation mechanism for off grid operations in the future.

In the meantime, I have developed something myself that will monitor battery usage and solar power generation, so that I can determine when to turn on the solar and when to turn it off. Note that this is only for off grid operations. Once we have the net-meter, we can go back to on grid operations, and the convenience of the grid can act as the main regulator of power.

However, this is excellent experience as it teaches us some of the off grid challenges. There is no substitute for living through the experiences.

We hope the net-meter will arrive soon. Until then, we will challenge ourselves to see how many days we can stay off grid! You can already see our progress on the 23rd and the 24th of this month from the above Alectra utilization chart.

Cheers! Until the next update.

Solar Panels on the Roof

On March 31st, I saw the following picture from my garage security camera:

The Panels Have Arrived!

It took several days to install the 56 335W panels from Canadian Solar. We just finished the installation yesterday and we are now awaiting for ESA inspection and commissioning the system.

The weather was borderline cooperative, being wet and fairly high wind situations (gusting near 50 to 100 km/h at times). However the installers from New Dawn Energy Solutions soldiered on and completed the installation yesterday.

Instead of me blabbing about how the install went, here is a short video on the near daily progress.

Near Daily Progress of Installation

The system is now ready to generate power as soon as ESA inspection is completed and our power meter is changed for net metering. Hopefully this will happen in the next week or so.

Residential Backup Battery Installed

In an earlier posting, I outlined how we initiated our solar panel project. Although the current weather condition prevents us from installing the solar panels at this point, we can install all of our required inverters and backup batteries.

On February the 22nd, we connected our Schneider Hybrid inverters to the grid, and on the 23rd, we connected the Pylontech LiFePO4 (LFP) batteries.

We configured the inverters so that the batteries will be discharged during the peak hours and charged during off-peak hours, effectively performing consumption time-shifting so that we can take advantage of the lower rates:

From Alectra Utilities

As you can see the savings are quite significant, more than 50%.

On February the 24th, was our first full day of usage when we tested our time-shifting configuration, and we found that it worked quite well. The battery capacity was enough to cover all of our on-peak hours usage save for the last remaining on-peak hour period.

Notice that we have more green in the off-peak hours because we are storing that capacity in the form of battery storage. I will play around with the configuration some more to see if I can shift the uncovered, on-peak hour to the mid-peak period, so that I have enough battery capacity left to cover all the on-peak periods.

I want to give a big shout out to New Dawn Energy Solutions. They have been very professional and really know their stuff. Any one thinking of installing a solar and/or a battery backup solution within the Greater Toronto Area, should seriously consider them. I highly recommend them and hope to do more business with them in the future.

Stay tune, and I will continue my progress here on the blog.