On March 31st, I saw the following picture from my garage security camera:
The Panels Have Arrived!
It took several days to install the 56 335W panels from Canadian Solar. We just finished the installation yesterday and we are now awaiting for ESA inspection and commissioning the system.
The weather was borderline cooperative, being wet and fairly high wind situations (gusting near 50 to 100 km/h at times). However the installers from New Dawn Energy Solutions soldiered on and completed the installation yesterday.
Instead of me blabbing about how the install went, here is a short video on the near daily progress.
Near Daily Progress of Installation
The system is now ready to generate power as soon as ESA inspection is completed and our power meter is changed for net metering. Hopefully this will happen in the next week or so.
In an earlier posting, I outlined how we initiated our solar panel project. Although the current weather condition prevents us from installing the solar panels at this point, we can install all of our required inverters and backup batteries.
On February the 22nd, we connected our Schneider Hybrid inverters to the grid, and on the 23rd, we connected the Pylontech LiFePO4 (LFP) batteries.
We configured the inverters so that the batteries will be discharged during the peak hours and charged during off-peak hours, effectively performing consumption time-shifting so that we can take advantage of the lower rates:
From Alectra Utilities
As you can see the savings are quite significant, more than 50%.
On February the 24th, was our first full day of usage when we tested our time-shifting configuration, and we found that it worked quite well. The battery capacity was enough to cover all of our on-peak hours usage save for the last remaining on-peak hour period.
Notice that we have more green in the off-peak hours because we are storing that capacity in the form of battery storage. I will play around with the configuration some more to see if I can shift the uncovered, on-peak hour to the mid-peak period, so that I have enough battery capacity left to cover all the on-peak periods.
I want to give a big shout out to New Dawn Energy Solutions. They have been very professional and really know their stuff. Any one thinking of installing a solar and/or a battery backup solution within the Greater Toronto Area, should seriously consider them. I highly recommend them and hope to do more business with them in the future.
Stay tune, and I will continue my progress here on the blog.
This year like last year is another quiet holiday period. The persistence of the Covid-19 virus and its many variants, especially Delta and Omicron, has made holiday travel near impossible and unpredictable.
Other than getting our Pfizer vaccine booster on Christmas Eve, there is not much to do then to treat myself to another smart home improvement project. The smart switches have really come down in prices. With my previous experience on the Homebridge server, it is now possible to acquire non-HomeKit compliant switches on the cheap and installed them with Homebridge so that I can still get Siri and Home App access.
This is exactly what I did on Boxing Day. I ended up purchasing the following switches and smart lock from Amazon:
I used some extra TD Travel reward points that I am not going to use anytime soon on the smart lock.
The two HS220 dimmers are used for the pot lights in my basement. The simple HS200 switch is hooked up to my chandelier. The TopGreener 3 way dimmer will be used for my basement landing light underneath the stairs, and finally the smart lock will be used in the door between the laundry room and the garage. I already have the same August lock for the front door. With the above additions, all the light switches are now automated in the shared living areas of the house.
All of the above devices are on its separate Wi-Fi and are bandwidth limited as well as isolated from the our internal LAN.
I am looking forward to reconfigure my HomeKit Automations with the above capabilities.
Update: I had to return the TOPGREENER. My basement landing light wiring had both switches at the end of the load, so I don’t have a close loop access to constant power, while the load / light is off. In effect, I thought I had access to a neutral wire, but it was just a piece of white wire connecting the hot lead. This just took 2.5 hours for me to figure it out.
I was couch surfing YouTube and came across this video:
Lawrence Technology Services detailing on the impact of Java Log4j
Since Unifi is my front line defence, I learned from the video that a patch from Unifi is already available for my Unifi Dream Machine (UDM) Pro. Of course, this prompted me to jump from my couch to my computer and immediately proceed to upgrade my Network application.
Thanks to Unifi, for addressing this so quickly. However, it was a release candidate so I could not upgrade it via the user interface. Instead I followed Unifi’s recommendation and did the following.
First remote shell via ssh into the UDM Pro.
% ssh udmpro
Welcome to UbiOS
By logging in, accessing, or using the Ubiquiti product, you
acknowledge that you have read and understood the Ubiquiti
License Agreement and agree to be bound by its terms.
___ ___ .__________.__
| | |____ |__\_ ____/__|
| | / \| || __) | | (c) 2010-2021
| | | | \ || \ | | Ubiquiti Inc.
|______|___| /__||__/ |__|
|_/ http://www.ui.com
Welcome to UniFi Dream Machine!
# unifi-os shell
root@ubnt:/# cd /tmp
root@ubnt:/tmp# ls
hsperfdata_unifi local.list unifi-network-status-response uploads
Ensure there is no previous version of unifi_sysvinit_all.deb in the /tmp directory. If so, then remove it.
The next step is to download the release candidate version via curl, and then install it using dpkg.
root@ubnt:/tmp# curl -o "unifi_sysvinit_all.deb" https://dl.ui.com/unifi/6.5.54-3b5d40203c/unifi_sysvinit_all.deb
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 126M 100 126M 0 0 27.1M 0 0:00:04 0:00:04 --:--:-- 28.7M
root@ubnt:/tmp# dpkg -i unifi_sysvinit_all.deb
(Reading database ... 65989 files and directories currently installed.)
Preparing to unpack unifi_sysvinit_all.deb ...
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76.)
debconf: falling back to frontend: Readline
Unpacking unifi (6.5.54-16676-1) over (6.5.53-16673-1) ...
Setting up unifi (6.5.54-16676-1) ...
Processing triggers for systemd (241-5~bpo9+1) ...
#-> ubnt-dpkg-cache install
removing /data/dpkg-cache/stretch/packages/unifi_6.5.53-16673-1_all.deb ... done
unifi: action=install, package=/data/dpkg-cache/stretch/packages/unifi_6.5.54-16676-1_all.deb mark=manual
<-# ubnt-dpkg-cache install
root@ubnt:/tmp# rm unifi_sysvinit_all.deb
Once the installation is competed, we remove the Debian package.
As you can see above, the Network application is now updated to 6.5.54.
Once again thanks to the folks at Unifi to come out with such a rapid fix.
Update:
I was curious whether my server is being attacked with this vector and sure enough there was an attempt logged at around 1pm today.
As I indicated in a previous post, I wanted to upgrade my old USG firewall to the new UDM Pro. In that post, I have outlined my reasons, so I will not repeat them here.
I wanted to perform the upgrade with a minimum amount of downtime. With several household members who are online constantly for both school work as well as for entertainment, any perceived downtime will result in a typical earful, which I would rather avoid.
In preparation for the upgrade, I performed the required backups for both the Unifi Video and the Unifi Network Controller applications that are both previously running on my NAS server. I copy the backups to an old MacBook Air that I will use as my console to setup the UDM Pro. I wanted to make sure that I stop both the unifi and unifi-video services on my Ubuntu NAS. We do not want to risk some sort of conflicts when we plug in the UDM Pro.
I also configured the old USG LAN2 port so that it will provision a network that has a separate IP subnet different from all of my existing networks. In my case, it was 10.10.11.1. I did this so that I can connect the UDM Pro into the LAN2 port of the old USG, which mimics my Internet Service Provider (ISP). This way, while I configure and setup the UDM Pro, my existing home network can continue to function and provide the require services to my household.
Connections used for UDM Pro setup
The setup connection layout looks like the above diagram. After the UDM Pro booted up, I used Safari to point to 192.168.1.1, which hosted the UDM Pro’s web administration interface. Since I already had a Unifi account, I used that to login initially. Once logged in, I proceeded to upgrade both the Network and Protect applications on the device. The Network application was upgraded to 6.5.53, and the Protect application was upgraded to 1.20.0. I am not planning to use the Access and Talk applications, but upgraded those any ways in case of any outstanding security holes.
I then open the Network application and performed a restore from the backup that I previously copied on to the MacBook Air. This restore worked without a hitch. All of my configurations and network and WiFi settings were ported over. The UDM Pro also automatically created another user from my old restore, which I promptly switched over, and disabled the external access to the UDM Pro, call me paranoid. I did a restart of the device to make sure that it boots back up smoothly. Once I have confirmed that everything is still okay, then I proceeded to shutdown the UDM Pro.
Before I make the physical swap, replacing the USG with the UDM Pro, I ssh into each Unifi managed device, switches and access points to ensure that the inform URL is reachable. Once I was satisfied of this, I then proceeded with the swap. Now my network looks like this:
Post upgrade layout (click to enlarge)
My expectation was that once the UDM Pro boots up and the Network application comes online, all the Unifi networking devices will automatically be managed by the UDM Pro. The reality however was slightly different. On the plus side, the network is fully functional and the only down time was the time it took to perform the swap and the boot up process of the UDM Pro, no more than 5 minutes. However on the down side, when I inspected all the devices (not clients) within the Network application, only one of all the devices successfully registered with the new Network application. I was a bit puzzled and miffed.
After about an hour’s investigation, apparently some devices did not like the inform URL to be http://unifi:8080/inform. I specifically picked the hostname instead of the IP address because it was changing from my NAS server to the UDM Pro. Unfortunately for some unknown reason, only a single Unifi access point UAP-AC-Pro made the jump and was successfully adopted by the UDM Pro. All the other devices went into a state of a repeated adoption loop. Even rebooting the devices did not help.
To remedy the situation, I had to ssh into each of the remaining devices and manually performed the following command (twice):
set-inform http://192.168.168.1:8080/inform
I am uncertain why I had to do it twice, but the first time did not work with the Network application. Once all the devices were adopted, I then proceed to configure the Override Inform Host settings on the UDM Pro. See below:
Note that the above settings were only in the New UI and missing form the Class UI
I had to set the above and not the default, because the will ubnt did not work, perhaps this had something to do with my Pi-hole configuration. Also, although by this point all the devices were connected, there were two remaining access points complaining about a STUN issue. When I check their respective logs, the STUN URL was incorrect because it retained the old unifi hostname. It was at this point, I decided to let the Network controller to push out an IP address based inform URL for all the devices.
I perform another reboot of the UDM Pro just to make sure we have transition stability over a power outage, and everything seems to be running smoothly now.
The restore of my Unifi Video configuration with the new Unifi Protect application on the UDM Pro was way more smoother. I dear say flawless. However, there was a minor hiccup. I run Homebridge at home to connect my Unifi Protect G3 Flex cameras to my HomeKit environment. This relied on an RSTP stream that was previously supported by Unifi Video. However the new Protect application used RSTPS, which is an encrypted version of the stream. Long story short, I had to switch from the Camera FFmpeg to the Unifi Protect plugin. Not a big deal, but I was super glad that the Unifi Protect plugin existed for Homebridge.
Now our inside security footages are recorded (up to a month), and motion detection is a lot faster. The application to find and view the videos are now more convenient. On top of that, live previews are also available within my Home App on my iPhone, iPad, as well as the Apple TV.
A gallery view from my 4K TV using Apple TV and HomeKit
The outdoor footages are still using Kuna, but I plan to perform a future upgrade using Unifi G4 Bullet cameras for the outside as well.
The last configuration I did was finally turned on Unifi’s Threat Management. I set it to level 4 to begin with and we may adjust it in the future.
click to enlarge
Last and certainly not the least, my NAS server is now more of a NAS and less of a networking controller, since all networking related monitoring and control is now being performed by a dedicated UDM Pro device. The UDM Pro can also handle the threat management at my full ISP 1Gbps download speed.
I will continue to post my upgrades here. If anything else, keep my future self informed.
We are all in with Unbiquiti’s Unifi lined of networking products. To date with the exception of a few unmanaged switches, all of our home networking switches and Wi-Fi access points are Unifi products.
Current Network Layout (click above to enlarge)
At the heart of our network is our USG Firewall (USG 3P). We’ve been using the secured gateway since December of 2017. During the holiday break, I plan to replace it with the new Unifi Dream Machine Pro (UDM Pro). The main reasons are:
The current USG 3P is limited to 85 Mbps when threat detection is turned on;
Run Intrusion Prevention System (IPS) and Intrusion Detection System (IDS) with the UDM Pro at full IPS speeds which for me is at 1 Gbps download and 30 Mbps upload;
Upgrade my current Unifi Video solution to Unifi Protect, since Unifi Video is no longer supported and I have five Unifi G3 Flex cameras;
Will most likely replace my current Kuna solution with Unifi Protect external cameras, so it is totally private, and my security video solution is unified;
The ability to connect Unifi Protect with my other devices using Homebridge;
The device has a capacity of up to 3.5 Gbps so we are future proofed for faster ISP speeds;
Move the UniFi Network Controller software that is currently running on my NAS server to a dedicated piece of hardware, the UDM Pro;
During my research with the UDM Pro, there is a slight regression in functionality that I currently use in my home. I currently use a functionality called static host mapping that allows me to give my NAS server several different host names so that I can use those host names to route to different in-home services, that is hosted by Apache2 server on my NAS. For example, the host name media.home routes to my home Plex server, and books.home routes to my Calibre server. Long story short, I need to run my own Domain Name Service to get this functionality back.
I have deployed and configured a DNSmasq installation before, but when I was exploring ways to reconfigure (hack) UDM Pro’s DNSmasq implementation, I came to the realization that it is probably best to offload the DNS functionality from the UDM Pro and run a dedicated instance on my NAS server. This way the upgrade path of the UDM Pro by Ubiquity in the future is nice and clean, and I can fully configure my own DNS the way I needed it.
With further research, I came across an alternative to DNSmasq, and that is Pi-hole with unbound, a full recursive DNS resolver. With this combination, I can essentially maintain my own local DNS records, satisfying my static host mappings, and also have a more private Internet experience for my entire home network when domain names are resolved by the authorized party.
I chose the Alternative 1 installation methodology on my NAS, but with a twist. Since I already have Apache2 installed, after the installation, I purged lighttpd from the installation, which came with the Pi-hole installation process. I also added the following to my Apache 2 configuration:
For extra security, I also added an .htaccess file containing:
% cat /var/www/html/.htaccess
<RequireAny>
Require ip 192.168.167.0/24
Require ip 172.17.168.1/24
</RequireAny>
The above ensures that only local computers or VPN clients on my network can gain access to Pi-hole. I also had to enable the above .htaccess file by adding the following in the main Apache 2 configuration file (/etc/apache2/apache2.conf):
<Directory /var/www/html/>
AllowOverride All
</Directory>
After a quick restart, Pi-hole is installed and operational. With an afternoon of testing, I noticed that some Google based shopping links have been blocked. We cannot have that during the holiday season, so I had to white list the following sites for them to work:
Click above to enlarge
Configuring unbound was really easy, I just had to follow the instructions here.
Now that I have my own DNS server running, I am no longer using Cloudfare or Google’s domain services. This translates to a little more privacy. I can also track which domains are being blocked and which domains are being accessed when I visit sites. Since this solution is site wide within my house, no additional work is required on iPhones, iPads, and client computers.
With this prerequisite step completed, I am now all set to migrate to the UDM Pro.
I thought it would be a good idea for me to give a a small tutorial on basic home networking issues, which many may find useful when diagnosing connectivity issues.
A modern, typical home network may look something like this:
Typical Network (click to enlarge)
Most people in our neighbourhood will have a cable based Internet access.1 Internet comes through via the coaxial cable, like the traditional cable that you used for your cable TV. This cable is connected to a cable modem, which in our case is the Hitron CDA3-35. The cable modem then makes the Internet accessible via classic networking cables with RJ45 plugs. Think of the cable modem as your main door to the Internet and nothing else. Since this box is typically provided by your cable company, you should probably not trust it, so it is a main door without a lock.
Some cable modems also do Wi-Fi, like the new Rogers Ignite Hubs. For best performance and better security,2 I would recommend configuring the cable modem in bridge mode and not in gateway or router mode. This means that it should not be the box provisioning and managing your network, and it will have its Wi-Fi functionality turned off. This also avoids double NAT-ing, something to be avoided in your home, in my opinion.3
You should invest in your own Wi-Fi access by purchasing something like the TP-Link AX1800 WiFi Router.4 This box provisions your residential network and your local Wi-Fi. You can purchase more advance / expensive Wi-Fi solutions here depending on the size and complexity of your residential layout.
If you have more than one Wi-Fi access point, I would recommend that they all have the same SSID but on different Wi-Fi channels. This will make it convenient and optimal for your Wi-Fi devices. Also keep in mind that some old / cheap IoT devices only like the 2.4GHz band. If you are in that situation, then you should create a specific 2.4GHz network with a different SSID.
If you want to try out VOIP (Voice-Over IP), you may also connect a VOIP Adapter. In our example, we have the Linksys PAP2T box. I am not going to go into details of how to acquire VOIP, or set it up, but this box effectively converts Internet traffic into voice traffic. Traditional landline phones can be linked up to the VOIP adapter using normal phone cables.
Okay, now that we have the different parts of the network defined, let us present a basic diagnostic workflow.
Basic Diagnostic Workflow
I hope the above introduction to the different parts of your home network and a workflow that you can follow will assist you in resolving some common connectivity issues in your home.
This spring, I installed solar panels on our green house. This project gave me the experience and knowledge of what I wanted for our house. In August of this year, we finally took the plunge and initiated our solar project for our house.
After much research, I settled with the following three vendors:
They all had a web presence and I initiated contact either by phone or with their online registration. For all three, I provided my postal code, my utility bill or usage, and they were able to prepare a quote for me to review. My initial request was for a grid-tie hybrid solution consisting of: Solar panels, and batteries. Specifically, I wanted to perform a full backup of my house electrical demands in the case of power outages. I wanted to avoid a typical solar only, net-metering, grid-tie solution. I also did not want a partial backup solution where certain high inductive loads such as air conditioners and dryers will not be available.
All three vendors came back with a simple solar net metering solution, the one that I specifically said I did not want. New Dawn Energy Solutions was the only vendor that gave me multiple options, one of which was a partial backup solution, which did not meet my full house backup requirement. With this initial misunderstanding, I thought it would be best that I spent sometime detailing exactly what my requirements are. I proceeded to create a slide deck with this purpose.
Long story short, getting a common understanding of my requirements was still a challenge for the vendors with the exception of New Dawn Energy Solutions. I was able to directly contact the engineer who prepared and designed the solution. This was during the weekend, and we were able to quickly clarify what I wanted and what New Dawn Energy Solutions can provide.
I decided to select New Dawn Energy Solutions and proceeded with a contract with them. While we await for permits, New Dawn Energy Solutions also helped me to start my energy audit for the Canada Greener Home Grant Program. Under this program, we can potentially get up to $5000 CAD back. The first of two audits was already performed by EnerTest. The auditor was super friendly, detailed, informative, and efficient. I would recommend EnerTest if you are going after the same program.
The current solution look something like this, but it is subject to change after an on site engineering assessment.
Our Solar Setup
As of this writing, the first energy audit is now completed. Now we will await for the engineering assessment and the required permits.
I am excited to generate clean energy and will no longer be guilty of enjoying the full capabilities of my air conditioner during the summer heat.
This is an update to my Sunroom Project that I detailed in a previous post.
After the installation of the solar panels, there was a question of whether the solar panels had enough juice to keep the batteries charge for cloudy days or night time operations. After a few days of operation, the observation is a definitive “no”.
The overall load of water pumps, fans, and temperature sensors amounted to be about 80W. The two solar panels (100W each, equaling 200W) during sunny days will only yield enough to cover this load. The less than optimal positioning on the sunroom’s roof will deprive the solar panels in operating in their optimal efficiency. Even on a really sunny day the panels may have a little surplus to give back to the battery, but nothing close to offer a continuous charge to the battery. Most of the time, the solar power is just enough to power the load and leave the battery as is. Left unattended, the batteries will slowly drain to nothing, as it is being discharged during cloudy days and at night times.
Automatic Transfer Switch from Amazon
So much for going entirely off grid! Since it is a sunroom, the roof based real estate is a bit precious, and two panels is as much as we wanted to take away from the sunshine that feed the plants. Not all bad news, we still have the opportunity to time shift our power needs from the grid, from peak time to non-peak time.
For the past few days, I had to go out to the sunroom during the evenings and switch the battery from solar power and back to the grid via the 480W DC Power Supply, so that it will charge itself back during the night. This is of course super inconvenient. What I needed is an Automatic Transfer Switch (ATS). Something like the one shown on the left offered by Amazon.
The price was pretty exorbitant, over $150.00, much more than what I wanted to spend. When there is a need, there is an opportunity to invent. I thought this is such a good opportunity to create my own ATS.
My ATS will be a simple 12V relay powered by the battery itself, and controlled by a WiFi capable microcontroller like the ESP32S that is perfect for the job. With a bit of searching on Amazon, I found this gem, a simple 12V relay that is only around $16.
URBEST 8 Pin JQX-12F 2Z DC 12V 30A DPDT
At most, the relay only needed to handle 10A, so the 30A rating is a total overkill, but good enough for my purpose.
I already have an ESP32S that I purchased earlier from AliExpress or Ebay. This is a WiFi enabled micro controller that can be programmed with the popular Arduino IDE. They were less than $5 a piece when I got them, and was literally sitting on my shelf awaiting for a project such as this. The master plan is as follows:
click to enlarge
The EPS32S will remotely control the relay, which physically makes contact between the battery with either of the solar controller or the power supply. The when is determined by a remote server on the same home network. The ESP32S will periodically post the battery voltage status and the current state of the relay to the server.
Using this approach, I can place the majority of the smarts on the server instead of the micro controller. I can also change the logic without having to reprogram the ESP32S.
The ESP32S comes with many GPIO pins. We will make use of three of the GPIO pins, one to detect battery voltage via a voltage divider. The other two will send a pulse to a simple latch that will drive the switch position of the relay. The latch will use a popular NE555 chip in bistable mode. Here is a simple schematic that I put together.
click to enlarge
I prototyped the above circuit on a breadboard and using a desktop power supply to simulate the battery.
Everything worked as expected, and I proceeded to solder everything up on a PCB.
Below is the Arduino sketch that I wrote for the ESP32S to report the battery voltage and the relay switch state to my server.
#include <WiFi.h>
#include <HTTPClient.h>
/* Server and WiFi configurations are fake */
#define SERVER_IP "192.168.1.5"
#ifndef STASSID
#define STASSID "##############-iot"
#define STAPSK "##################"
#endif
#define uS_TO_S_FACTOR 1000000
#define BATT_VOLTAGE_GPIO 36
#define GRID_PULSE_GPIO 25
#define SOLAR_PULSE_GPIO 26
float volt = -1.0;
RTC_DATA_ATTR int currentState = -1;
void setup() {
pinMode(BATT_VOLTAGE_GPIO, INPUT);
pinMode(GRID_PULSE_GPIO, OUTPUT);
pinMode(SOLAR_PULSE_GPIO, OUTPUT);
digitalWrite(GRID_PULSE_GPIO, LOW);
digitalWrite(SOLAR_PULSE_GPIO, LOW);
Serial.begin(115200);
Serial.printf("Connecting to %s..\n", STASSID);
WiFi.begin(STASSID, STAPSK);
while (WiFi.status() != WL_CONNECTED) {
delay(250);
Serial.print(".");
}
Serial.print("\nConnected with IP: ");
Serial.println(WiFi.localIP());
}
void pulse(int pin) {
digitalWrite(pin, HIGH);
delay(200);
digitalWrite(pin, LOW);
}
// the loop function runs over and over again forever
void loop() {
WiFiClient client;
HTTPClient http;
digitalWrite(LED_BUILTIN, HIGH);
float v = 0;
for (int i = 0; i < 200; i++) {
v += analogRead(BATT_VOLTAGE_GPIO);
delay(2);
}
volt = v / 200.0;
Serial.print("Voltage Read: ");
Serial.println(volt);
http.begin(client, "http://" SERVER_IP "/autoTransferSwitch.php"); //HTTP
http.addHeader("Content-Type", "application/x-www-form-urlencoded");
// start connection and send HTTP header and body
String postStr = "id=sunroom";
postStr += "&volt=" + String(volt);
postStr += "&state=" + String(currentState);
int httpResponseCode = http.POST(postStr);
digitalWrite(LED_BUILTIN, LOW);
Serial.printf("Response code: %d\nResult: ", httpResponseCode);
String instructions = http.getString();
Serial.println(instructions);
String result = "nothing";
long sleepTime = 5L;
char buffer[100];
strncpy(buffer, instructions.c_str(), 100);
const char d[2] = ",";
int field = 0;
char* token = strtok(buffer, d);
while ( token != NULL ) {
if (field == 0) {
result = String(token);
}
if (field == 1) {
sleepTime = atol(token);
}
token = strtok(NULL, d);
field++;
}
if (result.equals("solar")) {
pulse(SOLAR_PULSE_GPIO);
currentState = 1;
} else if (result.equals("grid")) {
pulse(GRID_PULSE_GPIO);
currentState = 0;
} else {
digitalWrite(GRID_PULSE_GPIO, LOW);
digitalWrite(SOLAR_PULSE_GPIO, LOW);
}
http.end();
Serial.print("Sleeping for seconds: ");
Serial.println(sleepTime);
esp_sleep_enable_timer_wakeup(uS_TO_S_FACTOR * sleepTime);
esp_deep_sleep_start();
}
One nice thing about the ESP32S is its ability to go into a deep sleep where it can keep its state with almost zero power. This way, the micro controller doesn’t act as a power sink for the entire system. I took advantage of this feature, so that the server can also tell the ESP32S how long it should sleep.
On the server side, I have a simple PHP script that will take into account time of day, and the current battery charge.
<?php
date_default_timezone_set('America/Toronto');
// I've changed the location to protect my own privacy
$lat = 42.9293;
$log = -102.9478;
$zenith = 90;
$nextWait = 900;
// The voltage levels are ADC readings from ESP32 (divided by 10) and not actual volts
// voltage level when battery is fully charged and can either be used or be charged with solar
$solarChargeLevel = 280;
// voltage level when battery is okay to be used with or without solar (come off of grid)
$okChargeLevel = 260;
// voltage level when battery must be charged (get on grid)
$mustChargeLevel = 252;
$now = time();
$sr = date_sunrise($now, SUNFUNCS_RET_TIMESTAMP, $lat, $log, $zenith);
$ss = date_sunset($now, SUNFUNCS_RET_TIMESTAMP, $lat, $log, $zenith);
$srStr = date("D M d Y - h:i:s", $sr);
$ssStr = date("D M d Y - h:i:s", $ss);
$logFile = "/home/kang/log/autoTransferSwitch.log";
$dateStr = date("Y-m-d H:i:s");
header("Content-Type: text/plain");
$id = isset($_POST["id"]) ? $_POST["id"] : null;
$batVolt = isset($_POST["volt"]) ? $_POST["volt"] : null;
// -1 = initial, 0 = grid state, 1 = solar state
$currentState = isset($_POST["state"]) ? intval($_POST["state"]) : -1;
file_put_contents($logFile, "$dateStr, " .
"device reported: battery voltage: $batVolt and current state: $currentState\n",
FILE_APPEND);
if (!is_null($id)) {
if ($id === "sunroom") {
$action = "nothing";
$b = intval($batVolt) / 10.0;
// During day is defined by one hour after sunrise and one hour after sunset
$duringDay = (($sr + 3600) <= $now && $now <= ($ss - 3600));
$duringNight = (!$duringDay);
if ($currentState == -1) {
if ($b <= $mustChargeLevel) {
$action = "grid";
$currentState = 0;
} else {
$action = "solar";
$currentState = 1;
}
} else if ($currentState == 0) {
// We are charging from the grid
if ($duringDay && $b >= $okChargeLevel) {
// During the day we want to use solar or the battery as much as possible;
// This is a trickle charge so that we can take advantage of the sun.
$action = "solar";
$currentState = 1;
} else {
// Otherwise charge until battery is full
if ($b >= $solarChargeLevel) {
$action = "solar";
$currentState = 1;
}
}
if ($b >= 0.985 * $solarChargeLevel) {
// We are getting closer to fully charge so
// reduce communication interval from 15 min to 5 min
$nextWait = 300;
}
} else if ($currentState == 1) {
// We are either using the battery or the solar panels
if ($b <= $mustChargeLevel || $now > $ss + 3600) {
// Charge the batteries if we must or one hour past sunset
$action = "grid";
$currentState = 0;
}
if ($b <= 1.025 * $mustChargeLevel) {
// We are getting closer to require charging so reduce
// communication interval from 15 min to 5 min
$nextWait = 300;
}
}
// for testing purpose
// $nextWait = 15;
echo ($action . "," . $nextWait);
file_put_contents($logFile, "$dateStr, $batVolt, $action, " .
"sun: [$srStr - $ssStr], new state: $currentState, wait: $nextWait\n",
FILE_APPEND);
}
}
The above state transition logic is pretty simple to follow so I am not going to explain it in depth here. There are a couple of features that I like to expand on.
By default, the sleep time is 15 minutes, but the server will shortened it to a shorter interval of 5 minutes when the battery is near empty or full. This sample frequency should be enough for the server to make the appropriate switching decision. Once a switch in the relay has occurred, the sleep time can be reverted back to 15 minutes. For debugging purposes, I can also change the server script to a very fast sample rate of every 15 seconds.
The other feature is the account of day and night time. This first version of the algorithm will attempt to use solar and/or battery during the day, and only charge from the grid when it is absolutely necessary. If we do charge from the grid during the day, we don’t need to fill it up, but only charge it to a level that can be used again. We will then attempt to top up the battery about 1 hour after sunset.
My ATS is now installed and operating for an entire day, so far so good and I don’t have to go into the sunroom any more to perform a manual switch. The algorithm can be further enhanced by getting additional readings from the solar controller, but I didn’t want to go through the trouble. I think what I have so far should be sophisticated enough. We’ll see.
2021-06-18 Update: After several days of operation, I noticed that the ATS, more specifically the ESP32 micro-controller hangs or fails to wake from deep sleep after a few hours of operation. Upon further investigation, it may be a combination of unstable supply voltage (from the battery), memory leaks of the standard WiFi libraries or the usage of String types. I am not sure. I had to re-write my ESP32 Arduino sketch to include a watch dog reset as well as perform a timed, software triggered hardware reset of the controller itself every 30 minutes or so. I also eliminated the deep sleep functionality and simply resorting to delays and resets. It has been running for about a week without any hiccups.
YouTube viewing has been one of our favourite pass times during the lock down nature of the Covid-19 pandemic. I personally have been watching quite a few channels on how to use LiPO4 cells to build rechargeable battery banks for solar applications, primarily for off grid purposes.
We have a sunroom in our back yard that we used during the summer to grow some vegetables. It has some electrical needs such as water pumps, a temperature sensor, and a fan. Currently there is an electrical socket, fed from the house, that we plug these devices into. We thought it would be a good project to try to get our sunroom off grid. This would be a good learning project.
The first task is to build a 12V LiFePO4 prismatic cells battery bank. I purchased 4 3.2V 100Ah battery cells from AliExpress. The cells came with bus bars so I did not have to purchase those. However, I did have to buy a battery management system (BMS) to balance and manage the charging and discharging of the battery cells. It was very tempting to buy a BMS from AliExpress, but I decided to be cautious and purchased one from a US vendor with the accompanying and preferred quality control. The company Overkill provides a 12V BMS specifically for four LiFePO4 battery cells in series.
It took a very long time for the batteries to arrive from China. I suppose the pandemic could be one of the many reasons for the delay. Once they arrived, I connected in parallel and proceeded to perform a top balance procedure with my voltage limiting desktop power supply. This step is required because each cell will have a different voltage potential from each other. We want all the cells to have the same voltage potential to maximize the capacity that we will get from the aggregated 12V battery bank.
Cells in parallel being topped balanced at 3.65V until zero current
To top balance all the cells, first I hook up the cells in parallel and charge them at a constant voltage of 3.65V. The charge will continue until my desktop power supply shows zero amp going into the battery. This process took a very long time, almost 2 days.
Once the cells are balanced, I reconfigured the cells in series and proceeded to hookup the BMS and the pure sine wave 600W inverter I purchased from Amazon. I had to buy 4 AWG wire, once again from Amazon, because the 10 AWG wire that I purchased earlier was not going to be enough if I want to discharge the battery at 600W which is going to result in more than 50A of current at 12V. I used the remaining 10 AWG wire for solar controller and panel hookups. I also got some XT90 connectors so that I can easily plug/unplug the solar charge controller, solar panels, and potentially plugin charger. I will talk about the solar side some more later on.
All wired up. The yellow XT90 connector is to either a solar charge controller or an external DC charger
So now that we have the guts of our 12V LiFePO4 battery pack, we need to find a suitable home for this thing. My wife had an extra plastic filing box hanging around which is perfect for this.
A filing box is perfect to fit everything
Custom grommets and added a PC fan
I needed to drill some holes to fit a 12V 120mm PC fan for ventilation, and a couple of 2″ grommets so that we can pass plugs and connectors through the box. The fan will be powered by the inverter.
At this point we have ourselves a 1200Wh portable super battery pack that can power up to 600W of electronics, which will be great for road trips. If you plug a 20W iPhone fast charger and charge your phone, it can continuously charge for 60 hours (2.5 days). That is a lot of phones. If your MacBook Air ran out of juice on the road, then this battery pack can power a 45W charger for your MacBook Air for more than a day, and also charge your computer fully. Quite a handy thing to have for emergencies.
Doubles as a 1200Wh portable battery bank
For the solar panels, I purchased two Xinpuguang 100 W flexible solar panels from Aliexpress. They were about $1 / Watt, a pretty good deal. I hook the two panels together in series and got a Victron BlueSolar MPPT 75/10 solar charger to manage the charging of the batteries. The charge controller can accept a maximum of 75V and outputs a maximum of 10A.
The charge controller will automatically adjust the amperage and voltage to the battery bank as required ensuring optimal charging scenario. During a sunny day, it will run the sunroom load from the panels and any remaining current will goto charge the battery. At night, the battery will run the sunroom.
Today, we installed the entire setup. The battery is placed inside the green house to give it some precipitation protection.
The panels are latched to the roof of the green house, one on each side.
The BMS unit has a bluetooth connection and an iOS App. I can use my iPhone when in bluetooth range of the battery to see if the battery is being charged or discharged.
I took the following screen shot of the app today at around 5pm EDT. You can see that there is no current going into the battery and no current going out of the battery. This means the sun is powerful enough to run all the pumps and other electrical appliances in the sunroom. Pretty cool!
It is still too early to tell yet whether there is enough sun power to charge the battery and run the electrical devices in the sunroom in a sustainable manner. My current suspicion is that the two panels are just enough even on a full, bright, sunny day and at peak hours, to power devices and also provide surplus current to charge the batteries.
Here is my overall connectivity diagram:
We will let the system run for about a week to see if this is sustainable during the summer months or not. If not, then I will have to create an automatic transfer switch so that we can intermittently recharge the batteries during the evening with an optional 480W DC charger, which I also got from Aliexpress. This charger can operate between 0-24V and 0-20A. To charge the battery bank, I have set it to a constant voltage of 14.0V and allow the output current to flow unrestricted. This should charge the battery fully in a little over 4 hours from scratch.
Overall, I learned a lot from this project and what a great way to spend the pandemic indoors. This could be a precursor to a DIY Tesla Powerwall Project. We’ll see.
