• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

Justin Joyce

Practical tips and tutorials about software development.

  • Standing Invitation
  • Featured Posts
  • Latest
  • About

Dedupe a list in Vim

Posted Apr 20, 2023 โ€” Updated Jan 10, 2024

Tldr: use :sort u

Surprisingly often, I find myself needing to dedupe a list, usually when digging through logs. There are many ways to dedupe a list, but Vim might just be the fastest one.

Take this list of UUIDs I pulled when investigating a recent issue:

02b74975-e399-4268-83e6-b8b931
06c9b33e-195e-478f-ae3c-d0700c
2368cf5f-488f-4b58-86a0-b4468f
18b7733b-91e6-46af-8a44-942396
13db0651-08af-4e7e-8e69-00d2a9
06c9b33e-195e-478f-ae3c-d0700c
1821d0a4-9741-48bb-9d8a-dffab5
2368cf5f-488f-4b58-86a0-b4468f
1821d0a4-9741-48bb-9d8a-dffab5
2368cf5f-488f-4b58-86a0-b4468f

From a quick look, I can see that a few of those IDs are duplicated. Let’s run regular Vim :sort on that list to make it more obvious:

02b74975-e399-4268-83e6-b8b931
06c9b33e-195e-478f-ae3c-d0700c # Dupe 1
06c9b33e-195e-478f-ae3c-d0700c # Dupe 1
13db0651-08af-4e7e-8e69-00d2a9
1821d0a4-9741-48bb-9d8a-dffab5 # Dupe 2
1821d0a4-9741-48bb-9d8a-dffab5 # Dupe 2
18b7733b-91e6-46af-8a44-942396
2368cf5f-488f-4b58-86a0-b4468f # Dupe 3
2368cf5f-488f-4b58-86a0-b4468f # Dupe 3
2368cf5f-488f-4b58-86a0-b4468f # Dupe 3

Ok, it looks like 3 IDs are duplicated. Let’s run :sort u to get a unique list:

02b74975-e399-4268-83e6-b8b931
06c9b33e-195e-478f-ae3c-d0700c
13db0651-08af-4e7e-8e69-00d2a9
1821d0a4-9741-48bb-9d8a-dffab5
18b7733b-91e6-46af-8a44-942396
2368cf5f-488f-4b58-86a0-b4468f

There we go, 6 unique IDs.

Vim’s sort has a few other powerful options, like sorting by number (so 100 comes after 20), or sorting in reverse order, but I use this :sort u deduping functionality the most.

Filed Under: Productivity, Tips, Vim

Primary Sidebar

Recent Posts

  • Every Built-In Vim Color Scheme (with screenshots)
  • Reverse a string in Python
  • Meeting Cost Calculator
  • Vim find and replace
  • What makes an effective development team

Categories

  • Arrays (5)
  • Command Line (9)
  • Dates (3)
  • Featured (7)
  • Git (7)
  • Golang (5)
  • Javascript (8)
  • Productivity (8)
  • Projects (4)
  • Python (15)
  • Regex (2)
  • Ruby (3)
  • Shell (2)
  • Thoughts (2)
  • Tips (11)
  • Tools (3)
  • Tutorials (1)
  • Vim (4)

Archives

  • July 2024 (1)
  • February 2024 (1)
  • January 2024 (1)
  • December 2023 (1)
  • November 2023 (1)
  • October 2023 (4)
  • September 2023 (1)
  • August 2023 (2)
  • July 2023 (5)
  • June 2023 (3)
  • May 2023 (6)
  • April 2023 (5)
  • March 2023 (5)
  • February 2023 (10)
  • January 2023 (6)
  • December 2022 (7)

Copyright © 2025 ยท Contact me at justin [at] {this domain}

  • Privacy Policy