{"id":823,"date":"2018-01-09T00:41:02","date_gmt":"2018-01-08T23:41:02","guid":{"rendered":"http:\/\/julia2.lucaamore.com\/?p=823"},"modified":"2024-12-02T19:41:15","modified_gmt":"2024-12-02T17:41:15","slug":"ffdup-ultra-fast-and-light-portable-duplicate-file-finder","status":"publish","type":"post","link":"https:\/\/www.lucaamore.com\/?p=823","title":{"rendered":"ffdup &#8211; Ultra Fast (and Light) Portable Duplicate File Finder"},"content":{"rendered":"<p><a href=\"https:\/\/www.lucaamore.com\/wp-content\/uploads\/2018\/01\/lukelv_cyberpunk_neural_network_data_science_laboratory_with_a__5b34a77d-fb30-4a7d-8c01-996026b83fc5.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-large wp-image-1556\" src=\"https:\/\/www.lucaamore.com\/wp-content\/uploads\/2018\/01\/lukelv_cyberpunk_neural_network_data_science_laboratory_with_a__5b34a77d-fb30-4a7d-8c01-996026b83fc5-1024x1024.png\" alt=\"\" width=\"584\" height=\"584\" srcset=\"https:\/\/www.lucaamore.com\/wp-content\/uploads\/2018\/01\/lukelv_cyberpunk_neural_network_data_science_laboratory_with_a__5b34a77d-fb30-4a7d-8c01-996026b83fc5.png 1024w, https:\/\/www.lucaamore.com\/wp-content\/uploads\/2018\/01\/lukelv_cyberpunk_neural_network_data_science_laboratory_with_a__5b34a77d-fb30-4a7d-8c01-996026b83fc5-300x300.png 300w, https:\/\/www.lucaamore.com\/wp-content\/uploads\/2018\/01\/lukelv_cyberpunk_neural_network_data_science_laboratory_with_a__5b34a77d-fb30-4a7d-8c01-996026b83fc5-150x150.png 150w, https:\/\/www.lucaamore.com\/wp-content\/uploads\/2018\/01\/lukelv_cyberpunk_neural_network_data_science_laboratory_with_a__5b34a77d-fb30-4a7d-8c01-996026b83fc5-768x768.png 768w\" sizes=\"auto, (max-width: 584px) 100vw, 584px\" \/><\/a><a href=\"https:\/\/www.lucaamore.com\/wp-content\/uploads\/2018\/01\/lukelv_cyberpunk_neural_network_data_science_laboratory_with_a__5b34a77d-fb30-4a7d-8c01-996026b83fc5.png\"><br \/>\n<\/a><\/p>\n<p>Some years ago, needing to quickly identify duplicate files within Unix file systems residing on embedded devices or NAS (e.g., Synology, QNAP, D-Link), and dissatisfied with the tools I found online, I developed a command-line Perl tool with the following goals:<\/p>\n<ul>\n<li>Very lightweight and usable even on embedded devices (requires only the Perl 5.7.3 interpreter or higher, with no additional dependencies);<\/li>\n<li>Maximum efficiency on very slow disks or network drives.<\/li>\n<\/ul>\n<p>Thus, the first version of the free software <strong>ffdup<\/strong> was born, which I released under the GNU\/GPL v3.0 license on:<\/p>\n<p><strong><a href=\"https:\/\/github.com\/lookee\/ffdup\">ffdup &#8211; GitHub Repository Link<\/a><\/strong><\/p>\n<div class=\"group\/conversation-turn relative flex w-full min-w-0 flex-col agent-turn\">\n<div class=\"flex-col gap-1 md:gap-3\">\n<div class=\"flex max-w-full flex-col flex-grow\">\n<div class=\"min-h-8 text-message flex w-full flex-col items-end gap-2 whitespace-normal break-words [.text-message+&amp;]:mt-5\" dir=\"auto\" data-message-author-role=\"assistant\" data-message-id=\"c750a8a2-3a4c-4497-8553-4bfa5d3deb64\" data-message-model-slug=\"gpt-4o\">\n<div class=\"flex w-full flex-col gap-1 empty:hidden first:pt-[3px]\">\n<div class=\"markdown prose w-full break-words dark:prose-invert light\">\n<p>The project includes a Bash script for the parametric generation of a directory and file structure to support automated unit tests.<\/p>\n<p>In a future version, I plan to introduce the detection of duplicate directories.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<p><a href=\"https:\/\/www.lucaamore.com\/wp-content\/uploads\/2018\/01\/lukelv_steampunk_a_texure_by_circuit_cables_blinking_led_a_lot__a7ece2ba-aed7-406e-8491-b36508f6fb48.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-large wp-image-1698\" src=\"https:\/\/www.lucaamore.com\/wp-content\/uploads\/2018\/01\/lukelv_steampunk_a_texure_by_circuit_cables_blinking_led_a_lot__a7ece2ba-aed7-406e-8491-b36508f6fb48-1024x1024.png\" alt=\"\" width=\"584\" height=\"584\" srcset=\"https:\/\/www.lucaamore.com\/wp-content\/uploads\/2018\/01\/lukelv_steampunk_a_texure_by_circuit_cables_blinking_led_a_lot__a7ece2ba-aed7-406e-8491-b36508f6fb48.png 1024w, https:\/\/www.lucaamore.com\/wp-content\/uploads\/2018\/01\/lukelv_steampunk_a_texure_by_circuit_cables_blinking_led_a_lot__a7ece2ba-aed7-406e-8491-b36508f6fb48-300x300.png 300w, https:\/\/www.lucaamore.com\/wp-content\/uploads\/2018\/01\/lukelv_steampunk_a_texure_by_circuit_cables_blinking_led_a_lot__a7ece2ba-aed7-406e-8491-b36508f6fb48-150x150.png 150w, https:\/\/www.lucaamore.com\/wp-content\/uploads\/2018\/01\/lukelv_steampunk_a_texure_by_circuit_cables_blinking_led_a_lot__a7ece2ba-aed7-406e-8491-b36508f6fb48-768x768.png 768w\" sizes=\"auto, (max-width: 584px) 100vw, 584px\" \/><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Some years ago, needing to quickly identify duplicate files within Unix file systems residing on embedded devices or NAS (e.g., Synology, QNAP, D-Link), and dissatisfied with the tools I found online, I developed a command-line Perl tool with the following &hellip; <a href=\"https:\/\/www.lucaamore.com\/?p=823\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":1558,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_sitemap_exclude":false,"_sitemap_priority":"","_sitemap_frequency":"","footnotes":""},"categories":[57,32,12,166],"tags":[54,56,55,68,71,53,52],"class_list":["post-823","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-bash","category-freesoftware","category-perl","category-programming","tag-dlink","tag-duplicate-file-finder","tag-ffdup","tag-free-software","tag-perl","tag-qnap","tag-synology"],"_links":{"self":[{"href":"https:\/\/www.lucaamore.com\/index.php?rest_route=\/wp\/v2\/posts\/823","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.lucaamore.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.lucaamore.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.lucaamore.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.lucaamore.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=823"}],"version-history":[{"count":40,"href":"https:\/\/www.lucaamore.com\/index.php?rest_route=\/wp\/v2\/posts\/823\/revisions"}],"predecessor-version":[{"id":2391,"href":"https:\/\/www.lucaamore.com\/index.php?rest_route=\/wp\/v2\/posts\/823\/revisions\/2391"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.lucaamore.com\/index.php?rest_route=\/wp\/v2\/media\/1558"}],"wp:attachment":[{"href":"https:\/\/www.lucaamore.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=823"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.lucaamore.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=823"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.lucaamore.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=823"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}