Find duplicate words in a string python

I can see where you are going with sort, as you can reliably know when you have hit a new word and keep track of counts for each unique word. However, what you really want to do is use a hash (dictionary) to keep track of the counts as dictionary keys are unique. For example:

words = sentence.split()
counts = {}
for word in words:
    if word not in counts:
        counts[word] = 0
    counts[word] += 1

Now that will give you a dictionary where the key is the word and the value is the number of times it appears. There are things you can do like using collections.defaultdict(int) so you can just add the value:

counts = collections.defaultdict(int)
for word in words:
    counts[word] += 1

But there is even something better than that... collections.Counter which will take your list of words and turn it into a dictionary (an extension of dictionary actually) containing the counts.

counts = collections.Counter(words)

From there you want the list of words in sorted order with their counts so you can print them. items() will give you a list of tuples, and sorted will sort (by default) by the first item of each tuple (the word in this case)... which is exactly what you want.

import collections
sentence = """As far as the laws of mathematics refer to reality they are not certain as far as they are certain they do not refer to reality"""
words = sentence.split()
word_counts = collections.Counter(words)
for word, count in sorted(word_counts.items()):
    print('"%s" is repeated %d time%s.' % (word, count, "s" if count > 1 else ""))

OUTPUT

"As" is repeated 1 time.
"are" is repeated 2 times.
"as" is repeated 3 times.
"certain" is repeated 2 times.
"do" is repeated 1 time.
"far" is repeated 2 times.
"laws" is repeated 1 time.
"mathematics" is repeated 1 time.
"not" is repeated 2 times.
"of" is repeated 1 time.
"reality" is repeated 2 times.
"refer" is repeated 2 times.
"the" is repeated 1 time.
"they" is repeated 3 times.
"to" is repeated 2 times.

View Discussion

Improve Article

Save Article

  • Read
  • Discuss
  • View Discussion

    Improve Article

    Save Article

    Prerequisite : Dictionary data structure Given a string, Find the 1st repeated word in a string. Examples:

    Input : "Ravi had been saying that he had been there"
    Output : had
     
    Input : "Ravi had been saying that"
    Output : No Repetition
    
    Input : "he had had he"
    Output : he

    We have existing solution for this problem please refer Find the first repeated word in a string link. We can solve this problem quickly in python using Dictionary data structure. Approach is simple,

    1. First split given string separated by space.
    2. Now convert list of words into dictionary using collections.Counter(iterator) method. Dictionary contains words as key and it’s frequency as value.
    3. Now traverse list of words again and check which first word has frequency greater than 1.

    Python3

    from collections import Counter

    def firstRepeat(input):

        words = input.split(' ')

        dict = Counter(words)

        for key in words:

            if dict[key]>1:

                print (key)

                return

    if __name__ == "__main__":

        input = 'Ravi had been saying that he had been there'

        firstRepeat(input)

    Output:

    had

    Time Complexity: O(length(words))

    Auxiliary Space: O(length(dict))

    View Discussion

    Improve Article

    Save Article

  • Read
  • Discuss
  • View Discussion

    Improve Article

    Save Article

    Given a sentence containing n words/strings. Remove all duplicates words/strings which are similar to each others.

    Examples:  

    Input : Geeks for Geeks
    Output : Geeks for
    
    Input : Python is great and Java is also great
    Output : is also Java Python and great

    We can solve this problem quickly using python Counter() method. Approach is very simple.

    1) Split input sentence separated by space into words. 
    2) So to get all those strings together first we will join each string in given list of strings. 
    3) Now create a dictionary using Counter method having strings as keys and their frequencies as values. 
    4) Join each words are unique to form single string. 

    Python

    from collections import Counter

    def remov_duplicates(input):

        input = input.split(" ")

        UniqW = Counter(input)

        s = " ".join(UniqW.keys())

        print (s)

    if __name__ == "__main__":

        input = 'Python is great and Java is also great'

        remov_duplicates(input)

    Output

    and great Java Python is also

    Method 2:  

    Python

    s = "Python is great and Java is also great"

    l = s.split()

    k = []

    for i in l:

        if (s.count(i)>=1 and (i not in k)):

            k.append(i)

    print(' '.join(k))

    Output

    Python is great and Java also

    Method 3: Another shorter implementation:

    Python3

    string = 'Python is great and Java is also great'

    print(' '.join(dict.fromkeys(string.split())))

    Output

    Python is great and Java also

    Method: Using set() 

    Python3

    string = 'Python is great and Java is also great'

    print(' '.join(set(string.split())))

    Output

    is Python great and also Java


    How do I find a repeated word in a string?

    ALGORITHM.
    STEP 1: START..
    STEP 2: DEFINE String string = "Big black bug bit a big black dog on his big black nose".
    STEP 3: DEFINE count..
    STEP 4: CONVERT string into lower-case..
    STEP 5: INITIALIZE words[] to SPLIT the string..
    STEP 6: PRINT "Duplicate words in a given string:".
    STEP 7: SET i=0. ... .
    STEP 8: SET count =1..

    How do I find the most repeated words in a string in python?

    Use the Counter() function (which gives the frequency of words as a key-value pairs), to calculate the frequency (number of times the word has occurred) of all the words. Create a variable to store the maximum frequency. Loop in the above words frequency dictionary using the for loop.

    How do I find a repeated character in a string in python?

    First, we will find the duplicate characters of a string using the count method..
    Initialize a string..
    Initialize an empty list..
    Loop over the string. Check whether the char frequency is greater than one or not using the count method..

    How do I remove repeated words from a string in Python?

    1) Split input sentence separated by space into words. 2) So to get all those strings together first we will join each string in given list of strings. 3) Now create a dictionary using Counter method having strings as keys and their frequencies as values. 4) Join each words are unique to form single string.