While at Moose Jaw, we stayed at Temple Gardens Hotel & Spa. Carol did a dip in the hot springs pool and she claimed that the mineral water really helped her psoriasis. The morning breakfast buffet was delicious along with the accompanying, friendly service.
After breakfast, we visited the Tunnels of Moose Jaw, before heading off on our final leg to Calgary. There were three tunnel tours, but we picked the one associated with the Cold War. The 45 minutes guided tour brings you back to the 1950’s of the Cold War era where facilities in Moose Jaw was used to trained NATO and 5 EYES spies. They also talked about how the Canadian government at that time surveil social (leftest) politicians from Saskatchewan thinking that they were potential communist spies.
Our final drive to Calgary was largely uneventful other than that Medicine Hat was a much bigger town than I original thought. The final charging stop was in the middle of nowhere with just an accompanying Subway restaurant. At each charging stop, three in total on this trip, our new Windex cleaning process seems to be working with the bug infested windshields.
Just as we approach the outskirts of Calgary, the sky opened up as if the city was rolling out the “yellow carpet” for us. Check out the stunning pictures below.
As a bonus the car was washed with heavy rain fall just before reaching the city boundary.
This morning we had another Tim Horton’s breakfast with a 7am rise and setting out at around 8:30am after a 30min charge near the Tesla Superchargers at Polo mall.
I had to take a little detour to pick up a used stereo amplifier that I purchased, the SMSL DA9. I was looking for a balanced XLR input amplifier for my bedroom and this came up on Canuck Audio Mart. Hopefully it will sound good, but I will not be able to test it until I get home at the end of the month.
The above pickup was about a 30 minute detour, but we had beautiful weather for our 6 hours drive to Moose Jaw. The straight roads, clear skies, and Tesla Auto Pilot made the journey pleasant and relaxing.
There was an interesting Co-op stop in the middle of nowhere at Whitewood, Saskatchewan.
About 50 km from Moose Jaw, the Tesla auto wipers started to act up. I think it is the dead bugs build up on the windshield, and disoriented the front cameras that doubles as the rain sensor. I will have to wash the windscreen tomorrow morning before we set out again.
Another thing I noticed is that Saskatchewan highways had the speed markers too far to the right of the road. In a few cases, the Tesla failed to pickup the speed limit which was 110 kph, it instead erroneously assumed 80 kph. This impacted the cruise control until I applied a 30 kph offset in the Auto Pilot menu. If I did not have this offset, the car will revert back to 80 kph and I will be experiencing phantom breaking when cruising at 120 kph. Of course, this is a bit dangerous if there is a car following you too closely.
We had a nice Chinese dinner at James Chinese Restaurant, which is about a 5 minutes walk from our hotel. The food was okay, and when you are hungry, anything tastes good. Nonetheless, I will go back again.
After dinner we took a nice stroll towards Crescent Park. Here are some photos that we took.
Tomorrow we are going to visit the Tunnels of Moose Jaw, and then straight on to Calgary. We were going to goto Edmonton, but I think we’ll hit Edmonton on the way back, the same with Regina.
Today’s drive is our longest yet. We checked out of our AirBnB, and proceeded to a charge station to top up or batteries. This took about 30 minutes. Afterwards, we helped ourselves to a bagel breakfast to go at Tim Hortons. We ended up starting our journey at around 7am.
Today we actually crossed time zones from Eastern to Central, so we actually gained an hour.
The drive was uneventful and we are learning to stop at Supercharging stations at towns or cities with big populations so that there is either a Walmart or some big restaurant row.
There are two things which I find are critical assets for these long drives. The first is a good audio book, and the second is Tesla’s Autopilot / Auto Steering features. Both of these make the hours go by quick.
As usual, Carol’s video of the day gives a better experience than my wordy blog here.
Last night we stayed at the Quattro Hotel in Sault Ste. Marie. I really liked this hotel. Carol got her swimming in using their indoor pool, and we also got some laundry done with their guest laundry facilities. The place also came with complimentary breakfasts with real eggs and sausages!
We started our drive at 11am to Thunder Bay. The first half of the trip was met with beautiful weather. Northern Ontario has some really nice sceneries. We also got a bonus view, with some of the trees already changing into their fall colours.
The second half of the drive was pretty dangerous. It got super foggy on the highway, and we could not see more than 30 to 50 meters in front of us. It was like this for about 300 km. At our last charging stop, we had to use the GPS to guide us to get there.
After about three charge stops and numerous washroom breaks, at around 8pm we finally arrived at Thunder Bay. To our disappointment, most restaurants closed at 9pm here, so we settled for something simple, like McDonald’s for dinner.
Tonight we are staying at another AirBnB place. A very nice one bedroom suite with our own washroom. The host, Dawna was very nice and waited for our arrival. The room has a very “family” feel to it. It is as though we are staying with relatives.
Tomorrow, we plan to start early to Winnipeg.
Carol’s video is worth a million of my words, so here it is.
Day 4: Driving from Sault Ste. Marie to Thunder Bay
Looking out from the bow of the ferry from Tobermory to South Baymouth
Today we began or journey to Sault Ste. Marie. We start from Tobermory by hopping on the Chi-Cheemaun Ferry. The ferry ride itself accommodates mostly vehicles, such as cars, motorbikes, various recreational vehicles, vans, etc. I was impressed at how many vehicles it was able to carry in a single trip. There must have been 40 to 50 vehicles easily.
Driving into the ferry was also interesting, as there is two levels. The main level for vehicles that require extra height and motorbikes. The second level consists of mostly regular cars and SUV’s.
The ends of the ferry opens up to allow the vehicles through
The ferry took us to South Baymouth where we had our first road trip incident with our Tesla Model Y. Because the car was transported with the ferry, it lost its GPS position and had to reacquire the signal when we get off the ferry. Well, that is how it is suppose to work. The GPS signal was acquired, but the software map on the main dash failed to update the “current position”.
We tried the following to no avail:
Rebooting the dash;
Powering off and on with both a 3 minutes and 5 minutes wait;
Called Tesla support and they suggested the same with a final recommendation to drive to a Tesla Service Centre!
I am totally blown away that such a software glitch can exists on a Tesla. The fix probably just require a recycling of the GPS module. Unfortunately, there is no user-friendly way of doing this.
We gave up on our troubleshooting and drove from South Baymouth to Sault Ste. Marie without the Tesla navigation app. This is more of a handicap than I thought. The car fails to precondition the battery when we arrive at the supercharger so it takes a little longer to charge. We also miss the mapping functionality that tells us the locations of the on-route superchargers and how many were available. Good old Waze on our iPhone to the rescue, and some manual range calculations.
When we arrived at our hotel at Sault Ste. Marie, and with the help of the hotel Internet, I finally “risked” a software update on our Tesla. I had to use my iPhone’s personal hotspot and download the new software update for the Tesla. With fingers crossed and 30 minutes of unwanted and undesirable anxiety, the update downloaded successfully. Another bare knuckles 30 minutes wait, we were rewarded with a correct map and location on the dash once more!
The moral of the story is, when taking a long road trip, DO NOT transport the Tesla other than under its own power. Lesson learned! Those who are planning on taking a road trip, I highly recommend that you avoid ferries and trains that can transport your Tesla.
Tomorrow, we are off to Thunder Bay with a functional navigation system again.
Here is Carol’s video for the day.
Day 3: Taking the ferry to South Baymouth and on towards Sault Ste. Marie
Yesterday we started our journey to Calgary. Our first stop is Tobermory where we plan to checkout the Flowerpot Island and do some hiking on the Bruce Trails.
Charging at Owen Sound
We started our journey at around 1pm. The drive is quite relaxing especially with Auto Steer enabled with our Tesla Model Y. We stopped at Owen Sound’s Walmart to get some supplies and did a quick charge before proceeding to Tobermory. We wanted to have lots of battery capacity when we arrive at Tobermory because we were not sure if we can charge there. By the time we came back to our car from Walmart, the car charged from 55% to 87%. We continue our drive to Tobermory.
When we arrived to Tobermory, we had about 65% battery left. This should be enough, but out of curiosity we discovered there was fast charging available by ChargeQuest at the Tobermory Community Centre. For fun, we charged it back up to 80%. At Owen Sound, we paid $12.37 for 25.79kWh ($0.48/kWh), and at Tobermory with ChargeQuest we paid $6.58 for 14.29kWh ($0.46/kWh), which took 17 minutes. It looks like the rates are more or less the same.
We booked an AirBnB about 1.5km South of town. Our host Susan, was very responsive. The place was really nice and both Carol and I really enjoyed our two night stay there. I personally love the huge and spacious bathroom. The weather was kind of cool, so we did not make use of the large backyard or the patio. Check this place out if you are looking for a spot to stay in Tobermory.
As mentioned earlier we wanted to visit the Flowerpot Island, but when we found out the prices, we decided to forgo this idea. Below are the prices from Blue Heron Cruises.
Prices for two adults to Flowerpot Island – Drop Off Cruise
Instead we just went to the Fathom Five National Marine Park for a 4km hike along its trail. Below are some photos we took along the way.
So getting back to the Flowerpot Island, the photo below is taken from the shores of the hiking trail, and it is as close to the island we got. Not a total loss!
We sighted Flowerpot Island!
Tomorrow (September, 14) we will be taking the MS Chi-Cheemaun ferry to South Baymouth and onto Sault Ste. Marie!
Below are the lovely video summaries that Carol has made for our first two days of travel.
Day 1: From Richmond Hill to TobermoryDay 2: Adventures in Tobermory
The advent of the Large Language Model (LLM) is in full swing within the tech community since the debut of ChatGPT by openAI. Platforms such as Google Colab, and similar variants from Amazon and Facebook allows software developer to experiment with LLM’s. The hosted model of the data center based GPU’s makes training and refinement of LLM’s tolerable.
What about using LLM on a local computer away from the cloud?
Projects such as llama.cpp by Georgi Gerganov makes it possible to run the Facebook open sourced Llama 2 model on a single MacBook. The existence of llama.cpp gives hope on creating a desktop that is powerful enough to some local development with LLM’s away from the cloud. This post documents an experimental procedure in building a desktop machine using parts readily available from the Internet to see if we can do some AI development with LLM’s.
Below is a list of sourced parts from EBay, Amazon and CanadaComputers, a local computer store. All prices are in Canadian dollars and includes relevant taxes.
GDSTIME 7530 75mm x 30mm 7cm 3in 12V DC Brushless Small Mini Blower Cooling Fan for Projector, Sleeve Bearing 2PIN (sourced from Amazon)
$16.94
CORSAIR Vengeance LPX 64GB (4 x 32GB) DDR4 3200 (PC4-25600) C16 1.35V Desktop Memory – Black (sourced from Amazon)
$350.28
AMD Ryzen 7 5700G 8-Core, 16-Thread Unlocked Desktop Processor with Radeon Graphics (sourced from Amazon)
$281.35
Noctua NH-D15 chromax.Black, Dual-Tower CPU Cooler (140mm, Black) (sourced from Amazon)
$158.14
Asus AM4 TUF Gaming X570-Plus (Wi-Fi) ATX motherboard with PCIe 4.0, dual M.2, 12+2 with Dr. MOS power stage, HDMI, DP, SATA 6Gb/s, USB 3.2 Gen 2 and Aura Sync RGB lighting (sourced from Amazon)
$305.09
Samsung 970 EVO Plus 2TB NVMe M.2 Internal SSD (MZ-V7S2T0B/AM) (sourced from Amazon)
Miscellaneous 120mm case fans and cables purchased from CanadaComputers
$63.17
The total cost of the above materials is $2,062.87 CAD.
The Nvidia Tesla P40 (Pascal Architecture) specializes for Inferencing limited to INT8 based operations and does not support any FP related operations, so it may not be optimal for machine learning. However recent claims have been made that INT8 / Q8_0 quantization can yield some promising results. Let us see what our experimentation will yield once the machine is built.
A custom design 3D fan shroud has to be designed and 3D printed because the P40 does not natively come with active cooling. The P40 is originally designed to operate in a data center so cooling is provided by the server chassis. The custom shroud design is posted on Thingiverse and some photos of the finished shroud is shown below.
Note that M3 screws were used to secure the shroud to the P40 GPU card. The GDSTIME fan came with the screws.
I also made a mistake by initially getting a 1000W ATX power supply that ended not fitting the case, because the case is built for SFX and SFX-L power supplies. Lesson learned!
Once the machine is built I performed a 12 hours MemTest86+. It turned out that running the memory at the XMP profile was a bit unstable. I had to clock the memory back from its 3200MHz rating to 3000MHz.
After more than 12 hours with 3 passes.
The BIOS settings had to be configured so that Resize BAR is ON. This is required for the P40 to function properly.
Turn on Resize BAR
The next step is to install Ubuntu 22.04.3 LTS with Nvidia GPU and CUDA drivers. The latter was quite challenging. The traditional way of installing using the package manager did not work. The best way is to goto this site, and pick the run file like below:
Beside to use the runfile
The run file had to be run in recovery mode using the console because the installation will fail if an X11 window manager is running. Also all previous Nvidia drivers had to be removed and purged. The Ubuntu default installation process may have installed them.
A detail that was left out of the instructions is to set the appropriate shell paths once the installation is completed. The following changes were made with /etc/profile.d so that all users can benefit. If the login shell is using zsh, then /etc/zsh/zshenv has to be changed. Without this change, commands such as nvcc and other CUDA toolkit commands will not be found. The same is true for CUDA related share libraries.
$cat /etc/profile.d/cuda-path.sh
export CUDA_HOME="/usr/local/cuda"
if [[ ! ${PATH} =~ .*cuda/bin.* ]]
then
export PATH="${PATH}:/usr/local/cuda/bin"
fi
if [[ ! ${LD_LIBRARY_PATH} =~ .*cuda/lib64.* ]]
then
export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/usr/local/cuda/lib64"
fi
if [[ ! ${LD_LIBRARY_PATH} =~ .*/usr/local/lib.* ]]
then
export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/usr/local/lib"
fi
In this hardware configuration the AMD CPU has integrated graphics, and the P40 does not have any HDMI or DisplayPort connections. We need to change the X11 configuration so that it will only use the AMD CPU while dedicating the P40 GPU for CUDA based computation. The following configurations have to be made in /etc/X11/xorg.conf:
The BusId can be obtained using the lspci command and be sure to change any hexadecimal notations to decimal in the configuration file. Without this xorg.conf configuration, the Ubuntu desktop will not start properly.
When everything is done properly, the command nvidia-smi should show the following:
Fri Aug 25 17:33:31 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.86.10 Driver Version: 535.86.10 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 Tesla P40 Off | 00000000:01:00.0 Off | Off |
| N/A 22C P8 9W / 250W | 0MiB / 24576MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| No running processes found |
+---------------------------------------------------------------------------------------+
The machine is now ready for user account configurations.
A quick video encoding using ffmpeg with hardware acceleration and CUDA was performed to test the GPU usage. It was a bit of a challenge to compile ffmpeg with CUDA support. This is when I found out that I was missing the PATH configurations made above.
For good measure, gpu-burn was run for an hour to ensure that the GPU is functioning correctly.
Next step is to download and setup the tool chain for LLM development. We will save that for another posting.
We have had our 2023 Model Y for almost three months now and are very happy with the driving and ownership experience. Many of our family, friends, and neighbours have been asking about the vehicle.
I have always referred them to my blog here for our account of the Tesla purchasing and driving experience.
Now for a very shameless plug. Anyone who is interested enough to start the path of purchase, below is my referral code:
We are all set! Today, I received my 9 years Chinese Q2 Visa, which allows multiple entries with each entry lasting up to 180 days. Getting this visa requires some perseverance and a little sweat. I am going to document my experience here so others can benefit.
Pick the first bullet if you are starting a brand new application.
Pick the last bullet if you are want to edit a previous application.
The application is online and you will have to gather your present passport information. There are other information which you will need to consider:
Old passport with a previous Chinese Visa; You will need to provide a photo copy of this;
Choose whether you want a tourist (L) visa or Q1/Q2 for visiting family members. There are other visa’s such as commercial, work, student exchange, etc. If you are visiting China, then typically it is either L or Q2. For details of all Chinese visa types, click here.
An invitation letter from a relative in China that you are visiting if you are applying for a Q2 visa. There is a specific form that you or your relative need to fill out. If you are applying for a Q2, then the relative also need to provide a copy of their passport and their Chinese ID card;
Also have all your previous and current citizenship information or landing papers information;
Some pointers when filling out the application:
If you are born in China, make sure you provide your Chinese name in section 1.1E;
If you were ever a Chinese citizen, you will have to provide past Chinese nationality information;
If you are not, then you will also have to provide an explanation and proof that you are not a Chinese citizen;
For Canadian citizens, the National ID number (1.6B) should be left blank;
Ensure that Place of issue (1.7D) and Issuing authority (1.7E) matches your Canadian passport;
Your work history;
Your education history;
Your immediate family information including father, mother, spouse, and children;
Your travel companion information;
A photo of yourself that meets their specification. I had to scan and adjust the photo for the purpose of the online application;
The full application is eight pages long!
Once your application is completed online, you should print / save a copy in PDF format. The web site will automatically prompt you to make an appointment. You cannot make an appointment unless you have a completed application online. The appointments are made in 30 minutes interval, and the place is called the University Centre, located at 393 University Avenue, Toronto, ON. Suite 1501:
There is parking in the building, but you will have to enter from Centre Avenue. We now have been to this building three times, and we did not have any issues finding a free spot in the underground parking at the University Centre.
There is a check list from the website, so make sure you bring all the required documents and photos.
When you arrive at the centre, you will have to line up to have your documentation triaged or inspected. This queue will be between 15 to 45 minutes depending on how busy they are. I suggest that you arrive 30 minutes prior to your appointment if not earlier. There will be people who have appointments lining up outside of the centre before they open at 9am.
Once your documentation and application pass the first round inspection, you will receive a ticket number, which you will have to observe on their wall mounted monitors that are hung around the centre. When your number is called, your documentation is further inspected and this time processed. If everything is okay, then you will receive another payment ticket number. Once again you wait for this number to appear on their monitors. This time to provide payment for the application. In return for your payment, you will get a pickup slip for your passport and visa.
For whatever reason, if your application did not past their first inspection, you will have to modify the application online and re-queue for triage. There are counters available at their office. If you bring your laptop, you can make changes to your online applications, and have it printed at Staples on the ground level at a building South of the University Centre.
Including ourselves, we also observed many people having issues with the applications and have to make modifications. Since I did not have my laptop with me, I had to come back the next day with the rectified application. I feel sorry for the seniors there who were not technically savvy and are lost in this process. If you have a senior who wants to get a visa to China, they WILL NEED help from someone who can complete the online application.
Our total effort include:
Getting a visa photo taken (0.5 – 1 hour);
Filling the application online, the first application took some time, subsequent applications will go a lot faster (4 – 6 hours);
Obtaining the invitation letter and have it filled out (0.5 – 1 hour);
First visit to the visa centre (2 hours + 2 hours of travel);
Second visit to the visa centre (2 hours +2 hours of travel);
Third visit to the visa centre to pickup visa (0.5 hour + 2 hours of travel);
Roughly speaking it was (for us) about 4 half days contributed to this exercise, and this did not include getting our Canadian passports renewed to ensure that we have the maximum time frame on the Q2 visa.
Get your mind psyched so that you are not too stressed in this endeavour!
Last Friday, we took the Tesla out to Montreal. On this trip we will really get to test the Supercharger network.
We left Richmond Hill at around 7am with a full charge of 100%. While on the highway, I took full advantage of the basic Autopilot feature, specifically the Traffic Aware Cruise Control (TACC) and Auto Steering. The TACC worked really well, but there were two instances where it did brake for some reason that was not evident to me. The Auto Steering for the most part really take the edge off of driving long distances. However its forcefulness of always driving in the centre of the lane is a bit scary, especially when passing tractor trailers. Normally when I drive and pass a truck, I typically stay more towards the left of the lane. I needed up turning off Auto Steer while passing trucks at times. I hope Tesla can add this lane “bias” as a future software option.
I also got use to passing with the basic Autopilot. I did not purchase the Enhanced Autopilot which will perform automated lane changes. I think this was a good call, as the Auto Steer is already a bit scary for me. We will have to warm up to the automated driving features a little more before we really adopt the new technologies. The jerkiness has improved as I got the timing down between signalling the lane change and re-engaging the Auto Steer once I am on the new lane.
Our first charge was at Gananogue, Ontario at fast Tesla Supercharger.
Tesla Supercharger at Gananogue
This was a very convenient location, as it was close to a McDonald’s. The charger was very fast. The car reported that it was ready to continue on the journey before we finished our breakfast. The charge time was less than 25 minutes, and the cost was slightly above $21. We had 30% on the battery when we arrived, and we left with 85%, which was enough for me to drive all the way to my uncle’s place in Montreal. I arrived with 26% battery left.
I decided to top the car off to 80% so that we don’t have to worry about our state of charge during our stay in Montreal. We went to Cote Vertu mall to perform another charge at its supercharging station.
Tesla Supercharger at Cote Vertu Mall
So the charging station here seemed older, and the handle plugs seemed a bit vandalized. Our first attempt stopped at 50+% for some reason, and I had to return to the car from the mall to plug it in again at a different station, which worked fine until 80%. This was the only “weird” incident I had with the Superchargers throughout the entire trip.
On the way back, we stopped by at the Cornwall supercharger, which was not ideal, because it was pretty far off the highway (more than 3km). It turned out to be a blessing in disguise since there was traffic on HWY 401, so it was good timing. We only charged for less than 10 minutes, just so that we get it top off enough to get to Gananogue again.
Tesla Supercharger at Cornwall
I got back home with around 15% of battery left. I plan to have the car back at home with a minimalist comfortable state of charge as possible because after all, we have solar power to charge the car back up.
While in Montreal, we also played with our J1772 charging adapter that came with the car. This adapter allows us to use a non-Tesla charger. The “Electric Circuit by Hydro Quebec” chargers were peppered throughout the island of Montreal. We found a convenient cluster of 4 chargers near where we stayed (~2km), and slow charged (5kW) our car for $1/hour. We left our car there for about 3 hours and got about 15kWh (~20%+). The slower charge is better for the battery any ways, and we spent the hours at my aunt’s place enjoying congee and more food.
Electric Circuit J1772 charger at Parc Pierre-Elliot-Trudeau in Cote-St-Luc
Another good story to tell is while visiting the Rene-Levesque Park, it was fairly difficult to find parking. However, there were an empty spot reserved for electric vehicles, so I took it! While we walked the park, the Tesla was happily charging at 5kW, so cool. Okay it cost me $0.80 for the charge, so it was not completely free, but can’t complain.
We did not have the opportunity to test our CCS Combo 1 adapter, which I purchased from Tesla just before our Montreal trip.
Combined Charging System (CCS) 1 adapter
The above adapter will allow us to get fast DC charging from a non-Tesla supercharger. Last night, we found one at the Hyundai Canada office near Hwy 7 and 404. This place is close to my parent’s place. The adapter worked well, and we did less than $3 charge there at a respectable rate of 50kW. However being a newbie, I should have charged it up for at least $5, since this is the “minimum” charge. Silly me!
We stayed for about 10 minutes and got about 10% more on the battery.
75 Frontenac Drive – Hyundai Canada
With this new found experience and successful tests of different charging methodologies, I think we are good to go for a cross Canada return trip to Vancouver in September!
