FAQ
I want to a fixed length list-like container, it should have a sorted()-like function that I can use to sort it,I think there should also a function I can use it to detect whether the numbers of items in it reaches the length of the container , because if the numbers of items in it reaches the length(fixed) of the container,I want to process the data in it .Is there a container in Python like this ?If not, what base container should be used to implement such container?


the container is similar to queue ,but queue doesn't have a sort function

Search Discussions

  • Chris Angelico at Nov 29, 2013 at 2:03 am

    On Fri, Nov 29, 2013 at 12:54 PM, iMath wrote:
    the container is similar to queue ,but queue doesn't have a sort function

    It's either a queue that can be sorted, or a list with a length limit.
    You could fairly easily implement either, because in Python, anything
    can be subclassed. But I think possibly the easiest way is to simply
    turn your queue into a list when you want to work with it:

    import queue
    a=queue.Queue(10)
    a.put("asdf")
    a.put("qwer")
    a.put("zxcv")
    a.put("1234")
    sorted(a.queue)
    ['1234', 'asdf', 'qwer', 'zxcv']


    Though I don't know if this is what you meant. Putting too much onto a
    queue.Queue will block. Were you wanting it, instead, to discard
    entries? If so, which?


    ChrisA
  • iMath at Nov 29, 2013 at 2:04 am
    All in all,I want to first fill the container, then sort it and process all the contents in it
  • Chris Angelico at Nov 29, 2013 at 2:06 am

    On Fri, Nov 29, 2013 at 1:04 PM, iMath wrote:
    All in all,I want to first fill the container, then sort it and process all the contents in it

    Where does the length limit come in?


    By the way, a little context helps a lot with following a thread.


    ChrisA
  • iMath at Nov 29, 2013 at 2:32 am
    hey , you used
    sorted(a.queue)

    this means the queue.Queue() has an attribute queue ,but I cannot find it described in the DOC ,where you find it ?
  • Cameron Simpson at Nov 29, 2013 at 6:19 am

    On 28Nov2013 18:04, iMath wrote:
    All in all,I want to first fill the container, then sort it and process all the contents in it

    Automatically?


    It sounds like an I/O buffer, in the sense that a block buffered
    output stream does a write on buffer full.


    Something like this:


       class Chunkifier:


         def __init__(self, size, process):
           self.size = size
           self.buffer = []


         def append(self, item):
           self.buffer.append(item)
           if len(self.buffer) >= self.size:
             process(sorted(self.buffer))
             self.buffer = []


         def extend(self, items):
           for item in items:
             self.append(item)


       def burp(items):
         for item in items:
           print item


       burper = Chunkifier(10, burp)
       burper.extend( (9,8,7,6,5,4,3,2,1,2,3,4,5,6,7,8,9,8,7,6,5,4,3,2,1) )


    Totally untested, buyer beware, worth what you paid for it, etc...


    Cheers,
    --
    Cameron Simpson <cs@zip.com.au>


    It is necessary for technical reasons that these warheads be stored with
    the top at the bottom and the bottom at the top. In order that there may
    be no doubt as to which is the top and which is the bottom, for storage
    purposes it will be seen that the bottom of each head has been labelled
    with the word TOP. - Instructions for storing British nuclear warheads
  • MRAB at Nov 29, 2013 at 2:23 am

    On 29/11/2013 01:54, iMath wrote:
    I want to a fixed length list-like container, it should have a
    sorted()-like function that I can use to sort it,I think there should
    also a function I can use it to detect whether the numbers of items
    in it reaches the length of the container , because if the numbers of
    items in it reaches the length(fixed) of the container,I want to
    process the data in it .Is there a container in Python like this ?If
    not, what base container should be used to implement such container?

    the container is similar to queue ,but queue doesn't have a sort
    function
    This is Python. You don't have to base it on an existing container.


    Write a list of the methods it should have. If none of the existing
    classes seem suitable as a superclass, then don't bother, just write it
    from scratch. You can always use an existing container, such as a list,
    as part of its implementation.
  • Terry Reedy at Nov 29, 2013 at 2:31 am

    On 11/28/2013 8:54 PM, iMath wrote:
    I want to a fixed length list-like container, it should have a sorted()-like function that I can use to sort it,I think there should also a function I can use it to detect whether the numbers of items in it reaches the length of the container , because if the numbers of items in it reaches the length(fixed) of the container,I want to process the data in it .Is there a container in Python like this ?If not, what base container should be used to implement such container?

    the container is similar to queue ,but queue doesn't have a sort function

    For single thread use, subclass list, add a new .put method that checks
    the list size and either does self.append or self.sort(); process(list).


    For multiple threads, you need the queue module, which has extra
    features (and baggage). A PriorityQueue yields items in sorted order.
    Subclass it and override .put to either call the original .put or start
    a process thread.


    --
    Terry Jan Reedy
  • Steven D'Aprano at Nov 29, 2013 at 3:32 am

    On Fri, 29 Nov 2013 13:03:00 +1100, Chris Angelico wrote:

    On Fri, Nov 29, 2013 at 12:54 PM, iMath wrote:
    the container is similar to queue ,but queue doesn't have a sort
    function
    It's either a queue that can be sorted, or a list with a length limit.
    You could fairly easily implement either, because in Python, anything
    can be subclassed. But I think possibly the easiest way is to simply
    turn your queue into a list when you want to work with it:
    import queue
    a=queue.Queue(10)
    a.put("asdf")
    a.put("qwer")
    a.put("zxcv")
    a.put("1234")
    sorted(a.queue)
    ['1234', 'asdf', 'qwer', 'zxcv']

    Though I don't know if this is what you meant. Putting too much onto a
    queue.Queue will block. Were you wanting it, instead, to discard
    entries? If so, which?



    Unless the OP needs all the extra threading-related powers of the queue
    module, I'd just stick to a simple, single-threaded queue object. These
    easiest way to do that is with a deque, which is a double-ended queue.
    The threading Queue class is based on a deque, so this will be at least
    as fast:


    py> from collections import deque
    py> my_queue = deque([], 5) # maximum of five items
    py> my_queue.append(7)
    py> my_queue.append(3)
    py> my_queue.append(9)
    py> sorted(my_queue)
    [3, 7, 9]
    py> my_queue.append(2)
    py> my_queue.append(0)
    py> my_queue.append(4)
    py> my_queue
    deque([3, 9, 2, 0, 4], maxlen=5)
    py> sorted(my_queue)
    [0, 2, 3, 4, 9]






    --
    Steven
  • Gregory Ewing at Nov 29, 2013 at 4:53 am

    iMath wrote:
    the container is similar to queue ,but queue doesn't have a sort function

    You can use a list as a queue. If you have a list l, then
    l.append(x) will add an item to the end, and l.pop(0) will
    remove the first item and return it.


    Then you just need to check the length of the list before
    adding an item, and if it's full, do something to process
    the items first.


    You can encapsulate all this inside a class if you want,
    but that's optional.


    --
    Greg
  • iMath at Nov 29, 2013 at 12:33 pm
    it seems PriorityQueue satisfy my requirement here .


    BTW ,the Queue object has an attribute 'queue' ,but I cannot find it described in the DOC ,what it means ?
  • Mark Lawrence at Nov 29, 2013 at 2:57 pm

    On 29/11/2013 12:33, iMath wrote:
    BTW ,the Queue object has an attribute 'queue' ,but I cannot find it described in the DOC ,what it means ?

    Really? AttributeError: type object 'Queue' has no attribute 'queue'


    --
    Python is the second best programming language in the world.
    But the best has yet to be invented. Christian Tismer


    Mark Lawrence
  • iMath at Dec 2, 2013 at 11:41 am
    ? 2013?11?29????UTC+8??10?57?36??Mark Lawrence???
    On 29/11/2013 12:33, iMath wrote:

    BTW ,the Queue object has an attribute 'queue' ,but I cannot find it described in the DOC ,what it means ?


    Really? AttributeError: type object 'Queue' has no attribute 'queue'



    --

    Python is the second best programming language in the world.

    But the best has yet to be invented. Christian Tismer



    Mark Lawrence

    you can do a check by hasattr()
  • Ned Batchelder at Dec 2, 2013 at 11:58 am

    On 12/2/13 6:41 AM, iMath wrote:
    ? 2013?11?29????UTC+8??10?57?36??Mark Lawrence???
    On 29/11/2013 12:33, iMath wrote:

    BTW ,the Queue object has an attribute 'queue' ,but I cannot find it described in the DOC ,what it means ?


    Really? AttributeError: type object 'Queue' has no attribute 'queue'



    --

    Python is the second best programming language in the world.

    But the best has yet to be invented. Christian Tismer



    Mark Lawrence
    you can do a check by hasattr()

    Yes, a Queue object has a queue attribute:

    import Queue
    q = Queue.Queue()
    q.queue
          deque([])


    But you shouldn't use it. It's part of the implementation of Queue, not
    meant for you to use directly. In particular, if you use it directly,
    you are skipping all synchronization, which is the main reason to use a
    Queue in the first place.


    It should have been named "_queue". We'll add that to the list of PEP-8
    violations in the Queue module! :)


    --Ned.
  • Chris Angelico at Dec 2, 2013 at 12:04 pm

    On Mon, Dec 2, 2013 at 10:58 PM, Ned Batchelder wrote:
    Yes, a Queue object has a queue attribute:
    import Queue
    q = Queue.Queue()
    q.queue
    deque([])

    But you shouldn't use it. It's part of the implementation of Queue, not
    meant for you to use directly. In particular, if you use it directly, you
    are skipping all synchronization, which is the main reason to use a Queue in
    the first place.

    I should apologize here; when the OP said "queue", I immediately
    noticed that I could import that and use it, and mistakenly started my
    testing on that, instead of using the deque type. It's deque that
    should be used here. Queue is just a wrapper around deque that adds
    functionality that has nothing to do with what's needed here.


    ChrisA
  • Ned Batchelder at Dec 2, 2013 at 12:26 pm

    On 12/2/13 7:04 AM, Chris Angelico wrote:
    On Mon, Dec 2, 2013 at 10:58 PM, Ned Batchelder wrote:
    Yes, a Queue object has a queue attribute:
    import Queue
    q = Queue.Queue()
    q.queue
    deque([])

    But you shouldn't use it. It's part of the implementation of Queue, not
    meant for you to use directly. In particular, if you use it directly, you
    are skipping all synchronization, which is the main reason to use a Queue in
    the first place.
    I should apologize here; when the OP said "queue", I immediately
    noticed that I could import that and use it, and mistakenly started my
    testing on that, instead of using the deque type. It's deque that
    should be used here. Queue is just a wrapper around deque that adds
    functionality that has nothing to do with what's needed here.

    ChrisA

    Actually, I had a long conversation in the #python IRC channel with the
    OP at the same time he was posting the question here, and it turns out
    he knows exactly how many entries are going into the "queue", so a
    plain-old list is the best solution. I don't know quite where the idea
    of limiting the number of entries came from.


    --Ned.
  • Cameron Simpson at Dec 3, 2013 at 9:17 pm

    On 02Dec2013 07:26, Ned Batchelder wrote:
    Actually, I had a long conversation in the #python IRC channel with
    the OP at the same time he was posting the question here, and it
    turns out he knows exactly how many entries are going into the
    "queue", so a plain-old list is the best solution. I don't know
    quite where the idea of limiting the number of entries came from.

    For me, it was because he said "fill the queue" in the OP. I took
    that to imply that the queue had a capacity limit.


    Cheers,
    --


    It's bubble, bubble, toil and trouble in conductive polymer.
    - overhead by WIRED at the Intelligent Printing conference Oct2006
  • Cameron Simpson at Dec 3, 2013 at 9:19 pm

    On 04Dec2013 08:17, I wrote:
    On 02Dec2013 07:26, Ned Batchelder wrote:
    Actually, I had a long conversation in the #python IRC channel with
    the OP at the same time he was posting the question here, and it
    turns out he knows exactly how many entries are going into the
    "queue", so a plain-old list is the best solution. I don't know
    quite where the idea of limiting the number of entries came from.
    For me, it was because he said "fill the queue" in the OP. I took
    that to imply that the queue had a capacity limit.

    And then I check the source:-( He actually said "I want to a fixed
    length list-like container". That still sounds like a limit to the
    number of entries.


    Cheers,
    --
    Cameron Simpson <cs@zip.com.au>


    Success in software development depends on making a carefully planned
    series of small mistakes in order to avoid making unplanned large
    mistakes. - Steve McConnell, _Software Project Survival Guide_
  • Ned Batchelder at Dec 3, 2013 at 9:34 pm

    On 12/3/13 4:19 PM, Cameron Simpson wrote:
    On 04Dec2013 08:17, I wrote:
    On 02Dec2013 07:26, Ned Batchelder wrote:
    Actually, I had a long conversation in the #python IRC channel with
    the OP at the same time he was posting the question here, and it
    turns out he knows exactly how many entries are going into the
    "queue", so a plain-old list is the best solution. I don't know
    quite where the idea of limiting the number of entries came from.
    For me, it was because he said "fill the queue" in the OP. I took
    that to imply that the queue had a capacity limit.
    And then I check the source:-( He actually said "I want to a fixed
    length list-like container". That still sounds like a limit to the
    number of entries.

    Cheers,

    Sorry, I was unclear. When I said, "I don't know where the idea came
    from," I should have said, "the OP said he wanted a fixed-length
    contained, but I don't know why he thinks that."


    --Ned.
  • Cameron Simpson at Dec 3, 2013 at 11:23 pm

    On 03Dec2013 16:34, Ned Batchelder wrote:
    On 12/3/13 4:19 PM, Cameron Simpson wrote:
    And then I check the source:-( He actually said "I want to a fixed
    length list-like container". That still sounds like a limit to the
    number of entries.
    Sorry, I was unclear. When I said, "I don't know where the idea came
    from," I should have said, "the OP said he wanted a fixed-length
    contained, but I don't know why he thinks that."

    Me either.


    Unless this is a homework task: "make one of these".
    --
    Cameron Simpson <cs@zip.com.au>


    It's as if all of Scottish cuisine is based on a dare. - Saturday Night Live
  • iMath at Dec 4, 2013 at 12:47 am
    ? 2013?12?4????UTC+8??7?23?49??Cameron Simpson???
    On 03Dec2013 16:34, Ned Batchelder wrote:
    On 12/3/13 4:19 PM, Cameron Simpson wrote:

    And then I check the source:-( He actually said "I want to a fixed
    length list-like container". That still sounds like a limit to the
    number of entries.

    Sorry, I was unclear. When I said, "I don't know where the idea came
    from," I should have said, "the OP said he wanted a fixed-length
    contained, but I don't know why he thinks that."


    Me either.



    Unless this is a homework task: "make one of these".

    --

    Cameron Simpson <cs@zip.com.au>



    It's as if all of Scottish cuisine is based on a dare. - Saturday Night Live

    deque meets my requirement here :)

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedNov 29, '13 at 1:54a
activeDec 4, '13 at 12:47a
posts21
users9
websitepython.org

People

Translate

site design / logo © 2022 Grokbase