{"id":730,"date":"2019-06-01T17:48:26","date_gmt":"2019-06-01T21:48:26","guid":{"rendered":"https:\/\/blog.lufamily.ca\/kang\/?p=730"},"modified":"2024-02-15T14:00:40","modified_gmt":"2024-02-15T19:00:40","slug":"nvme-ssd-with-lvm-cache","status":"publish","type":"post","link":"https:\/\/blog.lufamily.ca\/kang\/2019\/06\/01\/nvme-ssd-with-lvm-cache\/","title":{"rendered":"NVMe SSD with LVM Cache"},"content":{"rendered":"\n<style>\n.wp-block-preformatted {\nfont-size: 75%\n}\n.wp-block-preformatted-super-small {\nfont-size: 50%\n}\n<\/style>\n\n\n\n<p>I have been a huge fan of Apple&#8217;s fusion drives. They are an excellent compromise for affordable mass storage while still able to give you SSD performance. The concept is simple pair a fast but small SSD drive with a large but slow and much affordable, mechanical HDD. You get good performance and have lots of storage without breaking the bank.<\/p>\n\n\n\n<p>I have falsely assumed that this capability only existed with Apple&#8217;s macOS operating system. This week I was pleasantly surprised to have discovered that LVM Cache can do more or less the same thing on Linux. This new found knowledge along with an excellent deal on a <a rel=\"noreferrer noopener\" aria-label=\"500GB NVMe Samsung 970 Evo Plus M.2 (opens in a new tab)\" href=\"https:\/\/www.amazon.ca\/Samsung-970-EVO-Plus-MZ-V7S500B\/dp\/B07M7Q21N7\/ref=asc_df_B07M7Q21N7\/?tag=googleshopc0c-20&amp;linkCode=df0&amp;hvadid=292936769917&amp;hvpos=1o1&amp;hvnetw=g&amp;hvrand=9621750910136407494&amp;hvpone=&amp;hvptwo=&amp;hvqmt=&amp;hvdev=c&amp;hvdvcmdl=&amp;hvlocint=&amp;hvlocphy=9000804&amp;hvtargid=pla-636013621600&amp;psc=1\" target=\"_blank\">500GB NVMe Samsung 970 Evo Plus M.2<\/a> drive gave me the itch to experiment this weekend with my NAS media server.<\/p>\n\n\n\n<p>The hardware was easy enough to install, but I had to move one of the existing SATA connection because the M.2 slot on the motherboard shared a PCIe bus with a pair of SATA connections. Luckily I bothered to check the motherboard manual, otherwise I would have been scratching my head while the server fail to boot.<\/p>\n\n\n\n<p>The software configurations were a bit more involved. Before I purchased the NVMe card, I did some experimentation with two external USB drives, one SSD and one HDD. I found this <a rel=\"noreferrer noopener\" aria-label=\"article (opens in a new tab)\" href=\"https:\/\/ahelpme.com\/linux\/lvm\/ssd-cache-device-to-a-hard-disk-drive-using-lvm\/\" target=\"_blank\">article<\/a> to be super helpful in configuring LVM Cache with my test drives. However, these configurations were not fully restored after a reboot. After many hours of research on the Internet, I found this <a rel=\"noreferrer noopener\" aria-label=\"article (opens in a new tab)\" href=\"https:\/\/superuser.com\/questions\/1366742\/lvmcache-failed-to-connect-to-lvmetad-on-debian-stretch-during-boot\" target=\"_blank\">article<\/a> indicating that my Ubuntu Linux distribution was missing the <span style=\"font-family: courier\">thin-provisioning-tools<\/span> package. I also had experimented between the two different cache modes that were available, <span style=\"font-family: courier\">writethrough<\/span> and <span style=\"font-family: courier\">writeback<\/span>. I found out that the write back mode was a bit buggy and did not sync the cache and the storage drive. Yet another <a rel=\"noreferrer noopener\" aria-label=\"article (opens in a new tab)\" href=\"https:\/\/bugzilla.redhat.com\/show_bug.cgi?id=1668163\" target=\"_blank\">article<\/a> to the rescue.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">lvchange --cachesettings migration_threshold=16384 vg\/cacheLV<\/pre>\n\n\n\n<p>I preferred the write back mode due to its better write performance characteristics. Apparently to fix the issue, I have to increase the migration threshold to something larger than the default of 2048 because the chunk size was too large.<\/p>\n\n\n\n<p>Here are the steps that I did to configure my existing logical volume (<span style=\"font-family: courier\">airvideovg2\/airvideo<\/span>) to be cached by the NVMe drive that I just purchased. I first have to partitioned the NVMe drive.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">Model: Samsung SSD 970 EVO Plus 500GB (nvme)\u2029 Disk \/dev\/nvme0n1: 500GB\u2029 Sector size (logical\/physical): 512B\/512B\u2029 Partition Table: gpt\u2029 Disk Flags:&nbsp;\u2029 \u2029\u2029 Number&nbsp; Start &nbsp; End&nbsp; &nbsp; Size &nbsp; File system&nbsp; Name &nbsp; &nbsp; Flags\u2029 &nbsp;1&nbsp; &nbsp; &nbsp; 1049kB&nbsp; 500GB&nbsp; 500GB &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; primary<\/pre>\n\n\n\n<p>Create an LVM physical volume with the NVMe partition that was created previously <span style=\"font-family: courier\">\/dev\/nvme0n1p1<\/span> and add it to the existing <span style=\"font-family: courier\">airvideovg2<\/span> volume group.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">sudo pvcreate \/dev\/nvme0n1p1\u2029\n\nsudo vgextend airvideovg2 \/dev\/nvme0n1p1<\/pre>\n\n\n\n<p>Create a cache pool logical volume and set its cache mode to write back and establish the migration threshold setting.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">\nsudo lvcreate --type cache-pool -l 100%FREE -n lv_cache airvideovg2 \/dev\/nvme0n1p1\n\u2029\u2029\nsudo lvchange --cachesettings migration_threshold=16384 airvideovg2\/lv_cache\n\nsudo lvchange --cachemode writeback airvideovg2\/lv_cache<\/pre>\n\n\n\n<p>Finally link the cache pool logical volume to our original logical volume.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">sudo lvconvert --type cache --cachepool airvideovg2\/lv_cache airvideovg2\/airvideo<\/pre>\n\n\n\n<p>Now my original logical volume is cached and I have gained SSD performance economically on my 20TB RAID setup for less than $200. Below is my final volume listing.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted wp-block-preformatted-super-small\">$ sudo lvs -a\u2029 &nbsp; LV &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; VG&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Attr &nbsp; &nbsp; &nbsp; LSize &nbsp; Pool &nbsp; &nbsp; &nbsp; Origin &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Data%&nbsp; Meta%&nbsp; Move Log Cpy%Sync Convert\u2029 &nbsp; airvideo &nbsp; &nbsp; &nbsp; &nbsp; airvideovg2 Cwi-aoC---&nbsp; 20.01t [lv_cache] [airvideo_corig] 0.01 &nbsp; 11.78 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 0.00 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;\u2029 &nbsp; [airvideo_corig] airvideovg2 owi-aoC---&nbsp; 20.01t &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;\u2029 &nbsp; [lv_cache] &nbsp; &nbsp; &nbsp; airvideovg2 Cwi---C--- 465.62g &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 0.01 &nbsp; 11.78 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 0.00 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;\u2029 &nbsp; [lv_cache_cdata] airvideovg2 Cwi-ao---- 465.62g &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;\u2029 &nbsp; [lv_cache_cmeta] airvideovg2 ewi-ao----&nbsp; 64.00m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;\u2029 &nbsp; [lvol0_pmspare]&nbsp; airvideovg2 ewi-------&nbsp; 64.00m &nbsp; &nbsp; &nbsp;<\/pre>\n\n\n\n<p>We can also use the command below to get a more detail listing.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted wp-block-preformatted-super-small\">sudo lvs -a -o+name,cache_mode,cache_policy,cache_settings,chunk_size,cache_used_blocks,cache_dirty_blocks<\/pre>\n\n\n\n<p>Upgrade completed. We&#8217;ll see how stable it is in the future.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I have been a huge fan of Apple&#8217;s fusion drives. They are an excellent compromise for affordable mass storage while still able to give you SSD performance. The concept is simple pair a fast but small SSD drive with a large but slow and much affordable, mechanical HDD. You get good performance and have lots &hellip; <a href=\"https:\/\/blog.lufamily.ca\/kang\/2019\/06\/01\/nvme-ssd-with-lvm-cache\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;NVMe SSD with LVM Cache&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":731,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[111],"tags":[93,94,92,28,6],"class_list":["post-730","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tech","tag-lvm","tag-lvm-cache","tag-nvme","tag-technology","tag-ubuntu"],"jetpack_featured_media_url":"https:\/\/blog.lufamily.ca\/kang\/wp-content\/uploads\/sites\/3\/2019\/06\/58101998096__14D5E8C8-BE3A-4703-8BC5-1BBB1A91F9E7.jpeg","jetpack_shortlink":"https:\/\/wp.me\/p7V6i8-bM","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/posts\/730","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/comments?post=730"}],"version-history":[{"count":16,"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/posts\/730\/revisions"}],"predecessor-version":[{"id":1029,"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/posts\/730\/revisions\/1029"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/media\/731"}],"wp:attachment":[{"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/media?parent=730"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/categories?post=730"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/tags?post=730"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}