Python string append

Question

I have a python method that takes a list of tuples of the form (string, float) and returns a list of strings that, if combined, would not exceed a certain limit. I am not splitting sentences to preserve the output length, but making sure to stay within a sentence length from the desired output length.

For example:
s: [('Where are you',1),('What about the next day',2),('When is the next event',3)]

max_length : 5
output : 'Where are you What about the next day'

max_length : 3
output: 'Where are you'

This is what I am doing:

l=0
output = []
for s in s_tuples:
   if l <= max_length:
     output.append(s[0])
     l+=len(get_words_from(s[0]))
 return ''.join(output)

Is there a smarter way to make sure the output word length does not exceed max_length other than stopping when the length is reached?

Answer 1

First, I see no reason to defer the breaking out of loop if the maximum length is reached to the next iteration.

So, altering your code, I come up with the following code:

s_tuples = [('Where are you',1),('What about the next day',2),('When is the next event',3)]


def get_words_number(s):
    return len(s.split())


def truncate(s_tuples, max_length):
    tot_len = 0
    output = []
    for s in s_tuples:
        output.append(s[0])
        tot_len += get_words_number(s[0])
        if tot_len >= max_length:
            break
    return ' '.join(output)


print truncate(s_tuples,3)

Second, I really don't like that a temporary object output is created. We can feed the join method with the iterator which iterates over the initial list without duplicating the information.

def truncate(s_tuples, max_length):

    def stop_iterator(s_tuples):
        tot_len = 0
        for s,num in s_tuples:
            yield s
            tot_len += get_words_number(s)
            if tot_len >= max_length:
                break

    return ' '.join(stop_iterator(s_tuples))


print truncate(s_tuples,3)

Also, in your examples, the output is slightly bigger than the set maximum of words. If you want the number of words to be always less that the limit (but still the maximum possible), than just put yield after checking against the limit:

def truncate(s_tuples, max_length):

    def stop_iterator(s_tuples):
        tot_len = 0
        for s,num in s_tuples:
            tot_len += get_words_number(s)
            if tot_len >= max_length:
                if tot_len == max_length:
                    yield s
                break
            yield s

    return ' '.join(stop_iterator(s_tuples))


print truncate(s_tuples,5)

Answer 2

One smarter way would be to break out of the loop as soon as you exceed max_length, that way you are not looping over the rest of the list for no reason:

for s in s_tuples:
    if l > max_length:
        break
    output.append(s[0])
    l += len(get_words_from(s[0]))
return ''.join(output)

Answer 3

Your code doesn't stop when the limit is reached. "max_length" is a bad name ... it is NOT a "maximum length", your code allows it to be exceeded (as in your first example) -- is that deliberate? "l" is a bad name; let's call it tot_len. You even keep going when tot_len == max_length. Your example shows joining with a space but your code doesn't do that.

You probably need something like:

tot_len = 0
output = []
for s in s_tuples:
    if tot_len >= max_length:
        break
    output.append(s[0])
    tot_len += len(get_words_from(s[0]))
return ' '.join(output)

Answer 4

what is max_length supposed to control? the total number of words in the returned list? i would have expected a max_length of five to only yield 5 words, not 8.

EDIT: i would keep two lists around since i think it's easy to read, but some might not like the additional overhead:

def restrictWords(givenList, whenToStop):
    outputList = []
    wordList = []
    for pair in givenList:
        stringToCheck = pair[0]
        listOfWords = stringToCheck.split()
        for word in listOfWords:
            wordList.append(word)
        outputList.append( stringToCheck )
        if len( wordList ) >= whenToStop:
            break
    return outputList

so for

testList = [ ('one two three',1),
             ('four five',2),
             ('six seven eight nine',3) ]

2 should give you ['one two three'] 3 should give you ['one two three'] 4 should give you ['one two three', 'four five']

Answer 5

If NumPy is available the following solution using list comprehension works.

import numpy as np

# Get the index of the last clause to append.
s_cumlen = np.cumsum([len(s[0].split()) for s in s_tuples])
append_until = np.sum(s_cumlen < max_length)

return ' '.join([s[0] for s in s_tuples[:append_until+1]])

For clarity: s_cumlen contains the cumulative sums of the word counts of your strings.

>>> s_cumlen
array([ 3,  8, 13])