Wi-Fi 6 Upgrade with HomeKit Headaches

I recently upgraded all my WiFi access points to the Unifi UAP-U6-LR and UAP-U6-Lite. This will elevate my home to Wi-Fi 6 capable.

This was extremely exciting as my 802.11ax capable devices can now get between 100Mbps to 400Mbps depending on where we are in the house. It seems even the 802.11ac devices got about a 30% speed bump.

As a result of this upgrade, two UAP-AC-M mesh and one UAP-AC-Pro access points were retired from my house. I don’t recommend buying these devices any more since the Wi-Fi 6 devices from Ubiquiti are way more capable with higher performance and increase range than their 802.11ac access points.

However, the honeymoon period did not last long. After about a week, HomeKit devices started to show the dreaded “No Response” labels. Specifically, I had connectivity problems with Leviton Smart Decora Dimmers. In the past, all I had to do was recycle the HomeKit device and it was all good. Another episode of HomeKit and Leviton dimmer switch nightmare was experienced and documented by my previous blog post.

In this particular instance, the Leviton dimmers were able to join the Wi-Fi network and I can validate that with the Unifi Controller software. However, our HomeKit App was not able to connect to the dimmer switches. It took me sometime to figure out that the dimmers were unreachable by other Wi-Fi clients, but was reachable by computers that were physically wired to our network.

I found out which access point the dimmer switches were connected to and ssh into the access point to see if I can ping those devices, and sure enough they were unreachable. Below is a screen capture of the ARP listing from the access point.

Normal ARP listing from the Wi-Fi Access Point

When the dimmers were unreachable, the HW address was set to 00:00:00:00:00:00. After rebooting the culprit access point, I was able to access the offline dimmer switches again from the HomeKit App.

In summary, when HomeKit devices are offline with the dreaded “No Response” labels, here are the following things to try:

  • Ensure that local DNS is working properly and caches are emptied so that the latest data are available;
  • Ensure the device itself has acquired a valid IP address that is from your network;
  • Ensure that the device is reachable from the HomeKit App, typically from your iPhone or iPad;
  • Back trace the physical upstream networking equipment that is connected to your HomeKit device such as switches and access points and see which requires rebooting;

Apple could improve the HomeKit experience by allowing users to perform a full backup of the HomeKit configuration and reset the Home and perform a restore. Unfortunately, the closest thing that I found was from the Home+ App, but they do not restore device connectivity just their configurations.

When HomeKit works, you are literally like god, able to command light and switches with your voice in your home. When it does not work, it is extremely difficult to debug, due to a lack of diagnostics and logging.

After this update, my current networking layout now looks like this:

2021 October Network Layout (Click to Enlarge)

Ontario Covid-19 Vaccine Receipts

This morning I found out that the Ontario Government has made official Covid-19 Vaccine Receipts available through their website (https://covid19.ontariohealth.ca).

To download your receipts, all you will need is your Health Card. If you are like me and took a photo of your Health Card on your phone, it may not be sufficient, because information from both the front and back of the Health Card is required.

The service will allow you to download a PDF document for each dose of Covid-19 vaccine that you have received. To make it more convenient, I use the Preview app on my Mac to aggregate the data from both doses into a single PDF document. I then attached the PDF document on my iPhone’s Notes App for easy access should I require to show proof of my vaccination in 2021.

I also placed the information on my secured NAS server so that it can be accessed by any devices that has a secret link from the Internet, and generated a QR code for easy access for any third party that requires my proof of vaccination.

In the end, it looks something like what is shown above/right.

I also created a Siri Shortcut on the phone for easy access.

Now I have to repeat and rinse for all members of the family.

Ultrasonic Cleaner for Bike Chains

Generic Chain Cleaner

I like riding my bike but not cleaning my bike. Unfortunately cleaning my road bike especially the drive train is a necessity. Of course the most difficult part, the chain, is notoriously difficult to clean correctly.

In the past I have tried chain cleaners that look like the one on the right. In short, they don’t work.

SRAM Powerlink

The next evolution is to adopt a chain like the SRAM Powerlink or the Connex link, which can be easily taken apart. I still have to manually scrub the chain and it seems like no matter how many times you scrub the chain, it is still super dirty. Finally I came across the following YouTube video:

The host uses an ultrasonic cleaner and his result was really impressive. I went to Amazon and got myself one.

Flexzion Commercial Ultrasonic Cleaner 2L

I took off the chain and put it in the ultrasonic cleaner with a “cap” full of Simple Green all-purpose cleaner from Canadian Tire with hot tap water. I then run the cleaner for 10 minutes. After the first cleaning, the chain already look pretty spectacular. I lift the chain and repositioned it in the cleaner and run it for another 10 minutes. Took the chain out, rinse thoroughly with my garden hose, and put it back on. Here is the result with no scrubbing:

Click above to enlarge

Another nice thing about using this technique is that while the ultrasonic cleaner is doing its job, you can scrub the bike down. This bike cleaning session is the easiest one yet.

In summary, I highly recommend that you get an ultrasonic cleaner!

Update: Someone asked about cleaning the ultrasonic cleaner. There was no issue whatsoever. The grease did not stick to the container, and all I had to do was pour the dirty liquid out and give it a quick rinse. That was it. Simple. I see others on YouTube use a ziplock bag to contain the chain and the detergent, but I opted not to do that.

The Second Jab

At this point in time, all of our family members have our first dose of the Pfizer vaccine, and we are awaiting our second dose. As the Delta variant of the Covid-19 vaccine makes its way around the world, numerous reports are indicating that those who only have their first dose are only 33% protected from this variant.

Of course knowing this fact creates an a certain anxiety and urgency to get our second dose. Although we all have our second dose already scheduled when we got our first dose, those original schedules are weeks away. York Region today from 8am started taking appointments for rebooking second doses. This is of course very welcoming.

Unfortunately, as expected the scheduling web site experience leave very little to be desired.

I visited the site at around 7:50am, and it told me that I was in line and I had a 10 minutes wait. At this point, it is very reasonable, since it officially starts at 8am. I took the screen shot above, and as you can see the waiting time hasn’t really gone down and we have already passed our initial 10 minutes. To make matters worse, at around the 5 minutes mark, it immediately presented the booking form.

My sons who visited the site at 8am sharp got a waiting period of 1 hour. You can see how discouraging for some can be. Never mind vaccine hesitancy, it is these types of frictional booking experience that probably also discourages people from getting their doses. I really don’t understand why it is taking the organization so long to get there act together. Perhaps I’m being too harsh.

On the plus side, I did finally manage to book all members of my family for our second dose, because the site did allow me to make multiple bookings without having to virtually re-queue.

Unifi Access Point Disconnected

I have the following Unifi networking setup:

  • Unifi Secure Gateway (USG-3P) running 4.4.55.5377096 firmware
  • 3 Unifi Access Points AP-AC-PRO running 4.3.28.11361 firmware
  • 3 Unifi AP-AC-M running 4.3.28.11361 firmware
  • Unifi Controller (6.2.25) running on Ubuntu
  • The AP’s are connected to Unifi Switches running 5.43.36.12724 firmware

The AP’s are separated into two AP Groups. One of the AP Groups contains the 3 AP-AC-PRO and one of the AP-AC-M. This latter group was the problematic group. The other AP Group which contains the remain 2 AP-AC-M continue to work flawlessly throughout incident. From here on when I reference an AP Group, it is the problematic group that contains the 3 AP-AC-PRO and 1 AP-AC-M.

It all started yesterday when I noticed that my WiFi was a bit slow in the backyard and I wanted to change my radio configurations on one of the AP-AC-PRO and one AP-AC-M in the same AP Group. After the provision, the AP went disconnected as shown by the controller.

I ssh into the problematic AP-AC-PRO and discovered in /var/log/messages that there were many instances of the following log entry:

syswrapper: [state is locked] waiting for lock

I attempted to reboot the device but the device remained in the same “locked” state.

Since it is inconvenient for me to physically reset the AP, I attempted to reset the device via ssh using the command:

syswrapper.sh restore-default

Unfortunately, this did not always work because it immediately just shows the same locking message:

syswrapper: [state is locked] waiting for lock

I had to reboot the device via the command line reboot. As soon as I can ssh into the device after reboot, I immediately execute the restore command as above. This took a lot of trial and error because my timing is often off. When I miss the window, I will get the lock message again. I find that my chances are higher if I first forget the device on the controller first.

A quick suggestion to the Unifi team. It would be nice that the restore-default command if not able to restore immediately due to the lock, would at least set a flag in persistent flash memory of the device so that on the next reboot it will perform the restore then. This feature will safe me A LOT OF TIME!

Once the device is reset to factory default I proceeded to reconfigure it to the WiFi networks that I had. Unfortunately, when I try to provision the changes (adding the re-adopted device back into its original AP Group that is associated with the WiFi networks), it went into the disconnected state again. To make matters worse, the other AP’s in the same AP Group started to misbehave. Some would go into a provisioning state and followed by a disconnected state, while others go into an adopting state. This is of course very unnerving and frustrating. However, this observation lead me to remember a previous episode that I experienced a few weeks ago.

When I updated the controller to 6.2 and upgraded the AP’s firmware, the AP exhibited a similar locking issue. The solution that I employed was to restore to factory default and re-adopt the device. However after readopting, I assigned the AP to a brand new AP Group which I associated with the original WiFi networks. Simply adding the device to the original AP Group did not solve the issue.

When I tried this solution yesterday, it did not work. The device continues to go into a disconnected state immediately after provisioning when I added to the new AP Group. After many hours and much experimentation, I decided to erase all the WiFi networks and the problematic AP Group. I recreated the WiFi networks, and created a new AP Group and proceeded to add each AP one by one (after a reset to factory default). In summary here are the final steps that got me out of this pickle:

  1. Forget all AP’s in the affected AP Group.
  2. Remove all WiFi networks from the AP Group.
  3. Delete the AP Group.
  4. Delete all the WiFi networks that was associated with the above AP Group.
  5. Re-create all the WiFi networks and associate with a brand new AP Group.
  6. For each AP, use ssh to reset to factory default, adopt, and add them one at a time to the AP Group using the controller web UI.
  7. Since there were four AP’s (3 AC-Pro and 1 AC-M), I waited until the AP is fully connected and can service WiFi clients before I continue with the next one.

I am documenting this so that I can share with Unifi support. This has happened twice now, and each time I spent multiple hours to try to get my WiFi network working. In these pandemic times, WiFi is as important as electricity and plumbing. Since Tier-1 support was unable to resolve this issue, waiting for Tier-2 support (around 24 hours) is a bit “hard to swallow”.

Any ways I am glad that I was able to resolve this and brought my WiFi networks back up and running with the 4 affected AP’s, FOR NOW. However I must admit, these two episodes have made me apprehensive of making configurations to these AP’s, thinking that the next provisions will result in many more lost hours.

I hope the Unifi team can use this information and see if there is an issue relating to AP Group provisioning, since this seem to have triggered the issue in both cases.

I See the Light!

Today my wife and I got our first dose of Pfizer BioNTech Covid 19 Vaccine after probably more than 14 months since the first lock down notice from 2020. Of course it is still too soon to declare victory, as our second dose is still scheduled 3 months from now.

On a related note, the Ontario government has also declared the AstraZeneca vaccine will no longer be available as a first dose: “the decision was made out of an abundance of caution.”

CBC Article on AstraZeneca

I can’t help but think that this sounds a bit contradictory to what the government was peddling a few weeks ago. It kind of makes you think whether their 3 months guidance between the two doses of mRNA based vaccines is warranted and backed up by facts or not.

Regardless, this is another case of inconsistent information provided by our institutions. It certainly will continue to chip away at the credibility of the same institutions.

I hope we learn something from episodes such as this. In the meantime, both of our sons, who are age 17 and 16 should also be eligible for Pfizer or Moderna soon. We are now just monitoring our local clinic schedules to see when they can be booked. Fingers crossed, and three months cannot come soon enough.

The story continues in the media:

More updates from the CBC

Recent Gout Attack

On the Wednesday morning of April the 14th, right after I took out the garbage, I noticed that I experienced some tightness when I was bending my left knee going up the stairs. As time goes by the knee begins to swell. By lunch time, it was swollen to the point that I can no longer bend my knee and I can feel a continuous throbbing pain around my left knee. I can also feel that the temperature around my knee has also been elevated. When I took some measurements with a touchless temperature sensor for measuring fevers, my left knee is like 2ºC more than my right knee. This is not my first gout attack, but it has been very long since my last one that I couldn’t even remember when I had it.

I decided to document my current experience, so that next time, I personally have something to recount. Therefore, this blog entry is more for myself than you the reader if you are reading this already.

I have had swollen joints due to gout in my ankle and my knee, always on my left side for some reasons. I wasn’t sure whether I should take Allopurinol or wait until the attack subsides. I seem to recall that in the past that I should not take any Allopurinol during an attack, but the recollection was pretty murky, and doctors are hard to come by these days, so I read the following article: Allopurinol Doesn’t Worsen Acute Gout Attacks, and decided to take Allopurinol in my medicine cabinet. Later on I found out from my family doctor and my local pharmacist that I may have been better off not taking it. What I can tell you is that it started to heal even with the taking of Allopurinol during the attack, so there is some truth to the above article.

I also decided to take Ibuprofen which is available over the counter to reduce the inflammation. The Ibuprofen worked within a couple of hours and really helped with the throbbing. In the end, there is not much to do but to wait and drink lots of water and let my system dissolve and eat away at the uric crystals in my knee.

I got some diclofenac sodium topical cream, which I applied on the affected area overnight. Unfortunately, I felt some skin irritation, and an over tightening or stretching of the skin situation. I had to wash it out in the morning and stuck to Ibuprofen. A single does of 400mg Ibuprofen didn’t do the trick. I had to double the dosage to 800mg.

My hope of the swelling going away in a few days was dashed on the first weekend when I knew that it was getting worse and not better. It was not until the second weekend (approx. 10 days into the ordeal) when I can finally begin to bend my knee. At this point I stopped taking the Ibuprofen and let it naturally take its course.

By the third week, I can bend my knee for most daily activities but going down the stairs and simple things like putting on my pants still created a sharp pain sensation.

This has been the longest gout attack that I have experienced. This time around, I noticed significant muscle atrophy in my left thigh and calf. I also noticed pull tendon or cramping sensations when going down the stairs. Fortunately these sensations went away as I started to do more walking in the latter part of the third week.

On day 22, I still have minor pain on my knee cap but I was able to walk so I did a 3.5 km walk with my wife in the evening of May 6th. Since there were no additional swelling the following day, I decided to do an indoor bike ride of 20km. Yesterday was another 3.5 km of walking and today another indoor ride.

As you can see from the Strava statistics, I am improving, so back in the saddle and back on the climb. Slowly but surely.

Checking for Vaccine Schedule Availability

It looks like York Region is beginning to ramp up our vaccine schedules. We are now awaiting our postal code to be eligible so that my wife and I can make an appointment. At the time of writing this post, unfortunately, our postal code is still not eligible.

We know some of our neighbours are checking the website, https://york.ca/covid19vaccine on a regular basis. Obviously, this is very tedious and frustrating. I decided to embark on a small project to detect our postal code from the web site. My goal is to write a script that can be executed from my NAS server every 30 minutes to see if it can find our postal code on the website. My first attempt was to use Python and Selenium. The API was a bit clunky and I found it to be somewhat unreliable and fraught with timing issues. I gave up on this approach.

I thought I give it another shot, but this time using Puppeteer. This was much more simpler to use, and I found it to be more friendly especially if you’ve worked with Browser based Javascript to begin with.

I wrote this simple node script called scrape.js:

#!/usr/bin/env node

const puppeteer = require('puppeteer');

const targetPostalCode = process.argv[2] || 'A2B';

console.log ('scraping for ' + targetPostalCode + '...\n');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('http://york.ca/covid19vaccine');

  const anchorSelector = 'div#Y_accor div.new h3.Y_accordion span.trigger a';
  await page.waitForSelector(anchorSelector);

  console.log ('received...\n');

  const result = await page.evaluate((anchorSelector) => {
    let res = [];
    document.querySelectorAll(anchorSelector).forEach(function(elem){
      if (elem.textContent.match(/.*50 Years of Age and Older.*/i) != null) {
        elem.parentElement.parentElement.parentElement.querySelectorAll("p").forEach(function(elem){
          if (elem.textContent.match(/.*postal codes.*/i) != null) { 
            const postalCodes = elem.parentElement.querySelector("strong").textContent;
            postalCodes.split(", ").forEach(function(pc) {
              res.push(pc.trim());
            });
          }
        });
      }
    });
    return res;
  }, anchorSelector);

  console.log(result);

  let found = false;
  let regex = new RegExp(targetPostalCode, "i");
  result.forEach(function(postalCode) {
    if (postalCode.match(regex) != null) {
        // Postal Code Found!
        console.log("We found: " + targetPostalCode);
        found = true;
    }
  });

  console.log('closing...');

  await browser.close();
})();

I placed this script in the directory /home/xxxxx/gitwork/covidMonitor. Note that I targeted “50 years of Age and Older“, because this is the age group that I am personally interested in. The A2B is also not my real postal code. It is used instead of my own private postal code.

Before I run the program, I had to install puppeteer in my working directory.

cd /home/xxxxx/gitwork/covidMonitor
sudo npm i puppeteer

Running the above scrape.js produces the following:

scraping for A2B...

received...

[ 'L0J',
  'L4B',
  'L4E',
  'L4H',
  'L4J',
  'L4K',
  'L4L',
  'L6A',
  'L3S',
  'L3T',
  'L6B',
  'L6C',
  'L6E' ]
closing. 

Of course A2B is not in the list. If it was, another line like:

We found: A2B

will be in the output.

I supplemented the above with a shell script program which I can later use to configure cron to be executed every 30 minutes. The shell script’s job is to monitor the website using scrape.js and if the postal code is found, then send a notification email.

#!/usr/bin/env zsh

cd /home/xxxxx/gitwork/covidMonitor

if [ $(./scrape.js | tee /home/xxxxx/log/covidPostalCodes.log | grep "found" | wc -l) -ge 1 ]
then

sendmail xxxxx@gmail.com <<@
Subject: Covid Vaccine Can Be Scheduled

Goto the following site to schedule your vaccine:

https://york.ca/covid19vaccine

@

fi

I could have included the email notification logic in the node script as well, but I figure I may want to check manually on an infrequent basis, in which case I simply run the scrape.js script on the command line.

Now all there is to be done is relax and wait, a thing that we have been doing a lot lately during the pandemic.

Chinese Mauritian Roots

I recently received a YouTube video from one of my uncles on the topic of the migration of Hakka (客家) Chinese from Mexian, China to Mauritius.

My maternal heritage is from Mauritius and I have made previous postings in this regard. You can simply do a quick search in this blog on the term, Mauritius, and you’ll see other related postings.

For keep sake, I’ve gathered the videos here for easy reference to share this knowledge with other relatives. It is also pretty amazing what you will find when searching for “Chinese in Mauritius” on YouTube.

Below are four videos which I personally found interesting. The first three videos are from Jeanette Lan’s YouTube Channel, kudos to her for sharing these videos. They are listed in chronological order of production.

Commemoration of the 178th anniversary of the Arrival of Indentured Immigrants to Mauritius (November 2nd, 2012)

A Journey to Our Roots

The Cradle of Hakka Culture

Hakka History, Cooking, and Rhymes with the Toronto Hakka Heritage Alliance

Darci Door Bell

We have a cat named Darci. During the summer time she likes to take strolls through our backyard, checking out other frequent visitors like rabbits, chipmunks, squirrels, birds, and the like. She does this on an extended leash so to avoid her chasing these visitors and get lost into the wild. If you are curious, then visit her on Instagram.

We of course turn on the air conditioner on most days of the summer, so when Darci wants to come back into the house, she sits by the backyard sliding door and await one of us to open the door for her. If we are busy, she waits patiently for quite some time. My wife came up with the door bell idea. Instead of scratching the glass pane of the sliding door, which is mostly silent, we will train her to hit a door bell, which will alert us to let her in.

This is how I was assigned the Darci Door Bell project. This is another excellent opportunity to create a gadget. My first thought is to create a Raspberry Pi with a camera that uses an AI image classifier on the backend. I thought this would be a good opportunity to get my feet wet in AI. However, when dealing with a camera, and a Raspberry Pi, we are now talking about power and the need to be “plugged in”. We wanted a gadget that is wireless and battery operated. A battery based solution will severely restrict the power budget, so back to a WiFi based remote door bell idea.

After more research, I decided to use an ESP8266 MCU with WiFi and a simple step-down buck converter to convert 9V to 3.3V. I had worked with the ESP8266 before when I was attempting to use it to create my garage door opener. That was over six years ago. ESP8266 just came out and has yet to be integrated into the Arduino platform. Today, it is now much easier to work with the ESP8266. Using the Arduino IDE, one can simply program the ESP8266 natively as an MCU without even requiring an Arduino board.

From a previous project, I built a small circuit using an USB FTDI interface that will accept the ESP8266 ESP-01 board. This way I can plug the ESP8266 using USB to my computer and program it with the Arduino IDE. I wrote the following sketch, and programmed the ESP8266 ESP-01 board.

 #include <ESP8266WiFi.h>
 #include <ESP8266HTTPClient.h>

 #define SERVER_IP "192.168.1.111"
 
 #ifndef STASSID
 #define STASSID "myssid"
 #define STAPSK  "adsjfkdajfioeujfngjhf"
 #endif
 
 void setup() {
 
   WiFi.begin(STASSID, STAPSK);
 
   while (WiFi.status() != WL_CONNECTED) {
     delay(250);
   }
 
   WiFiClient client;
   HTTPClient http;

   // configure traged server and url
   http.begin(client, "http://" SERVER_IP "/IOTHandler.php"); //HTTP
   http.addHeader("Content-Type", "application/x-www-form-urlencoded");
 
   // start connection and send HTTP header and body
   http.POST("id=darciButton");
 
   http.end();
 
   ESP.deepSleep(0);
 }
 
 void loop() {
   ESP.deepSleep(0);
 }
 

The above program is pretty simple. When the board wakes up, it connects to my WiFi and then send an HTTP-POST request to my server that will handle the request based on the id, which in this case, it will be darciButton. As soon as the request is sent, the MCU will go into a deep sleep.

Completed Circuit

This way, the door bell will be a physical button that is hooked to the RST pin of the ESP8266 causing the ESP8266 to boot and send this POST request and immediately goes to sleep. The deep sleep is important, because while the device is in this deep sleep mode, the current draw from the 9V battery is negligible (<< 1mA). The only time the battery is used is when the button is pressed and released, and the ESP8266 is sending out the POST request. Using this power conservation approach, it is my hope that the 9V battery will last for the entire year of operation.

I used my 3D printer to print a little box for the whole thing and gave it some packaging padding, making sure the switch is on the top and properly supported. The final result looks something like this.

Packaged in a box

We then created a cardboard top cover that snugly fit the box, so that the top cover can freely move up and down the containing, plastic box. The switch itself has a little spring to it that allows the top cover to return back to its original position after being pressed. The finished product looks like this:

Finished Product

The top cover has a nickel on there so that Darci has a good target to train on.

Remember the server receiving the POST request? Well, I did some simple php programming on the server so that once the POST is received, I send out another HTTP request. This time to the Homebridge server with the homebridge-button-platform plugin installed. Homebridge is something that I already have and use to connect all non-compliant HomeKit accessories so that I can use those accessories via Siri. With this new plugin, I can connect this custom WiFi button to the HomeKit ecosystem within the house.

In effect, whenever this button is pressed, my HomeKit service will register an accessory button being pressed, and I can program a HomeKit scene that gets executed. I programmed a scene to play an audio file on all of my HomePods in the house. The audio file plays an audible door bell twice which I previously added to my Music library.

The final reward as shown by Darci herself. Watch the video below:

Darci Approves!

Of course with the button hooked up to HomeKit, we can now use this wireless button to do whatever HomeKit can do, unlock doors, turn lights on or off, etc. We will explore those possibilities in the future.