FAQ
======================================
   # fetch those columns that act as lists but are stored as strings
   cur.execute('''SELECT refs, visits, downloads FROM visitors WHERE
counterID = %s and host = %s''', (cID, host) )
   data = cur.fetchone()


   ref = visit = download = []
   if cur.rowcount:
    # unpack data into variables
    (ref, visit, download) = data


    # retrieve long strings and convert them into lists respectively
    ref = ref.split()
    visit = visit.split()
    download = download.split()
   else:
    # initiate these values
    ref = ref
    visit = lastvisit
    download = ''


   refs = visits = downloads = []
   # add current values to each list respectively
   refs.append( ref )
   visits.append( visit )
   downloads.append( download )


   # convert lists back to longstrings
   refs = ', '.join( refs )
   visits = ', '.join( visits )
   downloads = ', '.join( downloads )


   # save this visit as an entry into database
   cur.execute('''INSERT INTO visitors (counterID, refs, host, city,
useros, browser, visits, downloads) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)
       ON DUPLICATE KEY UPDATE refs = %s, visits = %s, hits = hits + 1,
downloads = %s''',
       (cID, refs, host, city, useros, browser, visits, downloads, refs,
visits, downloads) )
=============================================


IAM STRUGGLING WITH IT 2 DAYS NOW AND I CANNOT GET IT TO WORK.


ALL I WANT IT TO DO IS JUST


1. RETRIEVE 3 COLUMNS THAT CONSIST OF 3 LONG STRINGS
2. CONVERT LONG STRINGS TO LISTS
3. ADD SOME CURRENT VALUES TO THOSE LISTS
4. CONVERT FROM LISTS TO LONG STRINGS SO I CAN STORE SUCCESSFULLY LIST
PYTHON DATATYPE TO MYSQL SCALAR STRING.


EVERYHTIGN I TRIED FAILED.

Search Discussions

  • Antoon Pardon at Nov 5, 2013 at 5:23 pm

    =============================================

    IAM STRUGGLING WITH IT 2 DAYS NOW AND I CANNOT GET IT TO WORK.

    ALL I WANT IT TO DO IS JUST

    1. RETRIEVE 3 COLUMNS THAT CONSIST OF 3 LONG STRINGS
    2. CONVERT LONG STRINGS TO LISTS
    3. ADD SOME CURRENT VALUES TO THOSE LISTS
    4. CONVERT FROM LISTS TO LONG STRINGS SO I CAN STORE SUCCESSFULLY LIST PYTHON DATATYPE TO MYSQL SCALAR STRING.

    EVERYHTIGN I TRIED FAILED.

    Don't start a new thread, where you just ask the same question, you already asked before.
    The answer will not be different just because you started a new thread.


    Did you already read the documentation of fetchone?


    --
    Antoon Pardon
  • Mm0fmf at Nov 5, 2013 at 5:33 pm
    EVERYHTIGN I TRIED FAILED.

    Maybe try some of the advice you have been given instead?
  • Steven D'Aprano at Nov 5, 2013 at 5:41 pm

    On Tue, 05 Nov 2013 19:06:25 +0200, Nick the Gr33k wrote:


    ALL I WANT IT TO DO IS JUST

    1. RETRIEVE 3 COLUMNS THAT CONSIST OF 3 LONG STRINGS 2. CONVERT LONG
    STRINGS TO LISTS
    3. ADD SOME CURRENT VALUES TO THOSE LISTS 4. CONVERT FROM LISTS TO LONG
    STRINGS SO I CAN STORE SUCCESSFULLY LIST PYTHON DATATYPE TO MYSQL SCALAR
    STRING.



    You cannot do this with Python. You should use a better language more
    suited to you. Try this:


    http://php.net/manual/en/tutorial.php




    I hope that you will be much happier with this, since you are struggling
    to get Python to work the way you want it to.




    --
    Steven
  • Antoon Pardon at Nov 5, 2013 at 5:53 pm

    Op 05-11-13 18:41, Steven D'Aprano schreef:
    On Tue, 05 Nov 2013 19:06:25 +0200, Nick the Gr33k wrote:

    ALL I WANT IT TO DO IS JUST

    1. RETRIEVE 3 COLUMNS THAT CONSIST OF 3 LONG STRINGS 2. CONVERT LONG
    STRINGS TO LISTS
    3. ADD SOME CURRENT VALUES TO THOSE LISTS 4. CONVERT FROM LISTS TO LONG
    STRINGS SO I CAN STORE SUCCESSFULLY LIST PYTHON DATATYPE TO MYSQL SCALAR
    STRING.

    You cannot do this with Python. You should use a better language more
    suited to you. Try this:

    http://php.net/manual/en/tutorial.php


    I hope that you will be much happier with this, since you are struggling
    to get Python to work the way you want it to.

    Steve, what went on in your mind? If Nikos follows you advise he will
    probably come here with all kinds of php questions.


    --
    Antoon Pardon
  • Nick the Gr33k at Nov 5, 2013 at 5:59 pm

    ???? 5/11/2013 7:41 ??, ?/? Steven D'Aprano ??????:
    On Tue, 05 Nov 2013 19:06:25 +0200, Nick the Gr33k wrote:

    ALL I WANT IT TO DO IS JUST

    1. RETRIEVE 3 COLUMNS THAT CONSIST OF 3 LONG STRINGS 2. CONVERT LONG
    STRINGS TO LISTS
    3. ADD SOME CURRENT VALUES TO THOSE LISTS 4. CONVERT FROM LISTS TO LONG
    STRINGS SO I CAN STORE SUCCESSFULLY LIST PYTHON DATATYPE TO MYSQL SCALAR
    STRING.

    You cannot do this with Python. You should use a better language more
    suited to you. Try this:

    http://php.net/manual/en/tutorial.php


    I hope that you will be much happier with this, since you are struggling
    to get Python to work the way you want it to.



    --
    Steven, i want switch to php, i uses to start with php, then perl and
    now i ended up with python.


    All this problem arises because MySQL's hasn't got a datatype able to
    store an array of elements, a list.


    Is there some other database, sql connector i can use except 'pymysql'
    that supports a "collection" record type?


    postgresql or somethign similar?
    If it does i will switch to that and avoid these tedius convertions from
    long strings => list and backwise.


    Somehting that can just take Python's datatypes 'list' or 'tuple' or
    'dict' as tehy are and just store them into the database convertionless.
  • Tobiah at Nov 5, 2013 at 6:11 pm
    All this problem arises because MySQL's hasn't got a datatype able to store an array of elements, a list.


    Um, yes it does. It's called a table.
  • Joel Goldstick at Nov 5, 2013 at 7:01 pm

    On Tue, Nov 5, 2013 at 1:11 PM, Tobiah wrote:
    All this problem arises because MySQL's hasn't got a datatype able to store
    an array of elements, a list.

    Um, yes it does. It's called a table.
    --
    https://mail.python.org/mailman/listinfo/python-list

    Perhaps we are splitting hairs, but a relational database will let you
    stuff any text you want in a text field, but First Normal Form is a
    necessary (and not complete) requirement (from wikipedia)


    First normal form (1NF) is a property of a relation in a relational
    database. A relation is in first normal form if the domain of each
    attribute contains only atomic values, and the value of each attribute
    contains only a single value from that domain.[1]


    If you don't have a single thing in a field, you can't search or sort
    it or filter by its value with sql. Since those are very important
    properties of sql, not using 1nf is similar to tossing a whole
    warehouse of stuff in a warehouse without any sort of organization for
    where different things are to be put, and therefore, where they can be
    found.


    If you don't have first normal form data, you are misusing a
    relational database. If you don't want to learn about relational
    databases and what they can do to help you construct software that
    solves interesting problems, then perhaps you are not curious enough
    to ever become competent in the field.


    That said, sql is a different beast than is python or php or any
    procedural (or oops) language. Just like writing HTML and CSS is
    something that people who write computer programs for the web might
    do, its a total shift in thinking from writing python.


    There is no rule that a person must be a computer programmer, but if a
    person wants to be a competent and successful computer programmer, one
    must learn from the evolving understanding of the last 50 years or so.
      Its a craft of understanding how to divide impossibly large problems
    into understandable pieces -- using the best tools for each piece.
    You may say this is just one person's opinion -- but then again, I'm
    not the one screaming in all caps about the same several problems over
    and over for the last year or more on this list. The results show no
    more understanding or growth in skills, and an attitude that never
    shows a glimmer of interest in learning.








    --
    Joel Goldstick
    http://joelgoldstick.com
  • Denis McMahon at Nov 5, 2013 at 6:02 pm

    On Tue, 05 Nov 2013 19:06:25 +0200, Nick the Gr33k wrote:


    IAM STRUGGLING WITH IT 2 DAYS NOW AND I CANNOT GET IT TO WORK.

    Try starting with something simple. The following is a step by step guide
    to working out how you need to do this. Follow all the steps. Do not skip
    any steps. Each stage builds on the previous code.


    Write python code to create a list of strings and print out the elements
    of the list. This just needs to generate a list of n entries of m
    characters in length of random character data.


    Write python code to concatenate a list of strings into some longer
    string using a separator and print out the long string.


    Now add some python code to split the long string into a list based on
    the separator, and print out the list elements.


    Once you have this code working, make a test table with a single string
    column and an integer index field in a test database.


    Now this is starting to push the boundaries of your abilities, but write
    code to create two lists, and store them with indexes 1 and 2
    respectively.


    Now, and this is really really going to tax you, write some more code
    that will first retrieve the string for index 1, convert it back into a
    list, and display the list contents, then do the same for list 2, then
    retrieve all the data from the db and reproduce each list in turn.


    Then you might be ready to try coding what you're trying to code.


    --
    Denis McMahon, denismfmcmahon at gmail.com
  • Nick the Gr33k at Nov 5, 2013 at 6:09 pm

    ???? 5/11/2013 8:02 ??, ?/? Denis McMahon ??????:
    On Tue, 05 Nov 2013 19:06:25 +0200, Nick the Gr33k wrote:

    IAM STRUGGLING WITH IT 2 DAYS NOW AND I CANNOT GET IT TO WORK.
    Try starting with something simple. The following is a step by step guide
    to working out how you need to do this. Follow all the steps. Do not skip
    any steps. Each stage builds on the previous code.

    Write python code to create a list of strings and print out the elements
    of the list. This just needs to generate a list of n entries of m
    characters in length of random character data.

    Write python code to concatenate a list of strings into some longer
    string using a separator and print out the long string.

    Now add some python code to split the long string into a list based on
    the separator, and print out the list elements.

    Once you have this code working, make a test table with a single string
    column and an integer index field in a test database.

    Now this is starting to push the boundaries of your abilities, but write
    code to create two lists, and store them with indexes 1 and 2
    respectively.

    Now, and this is really really going to tax you, write some more code
    that will first retrieve the string for index 1, convert it back into a
    list, and display the list contents, then do the same for list 2, then
    retrieve all the data from the db and reproduce each list in turn.

    Then you might be ready to try coding what you're trying to code.



    --
    Denis, i have already provided my code trying to do what i need and i
    need some commendation on how to make it work.


    O even better an rdbms than allows complex data such as tuples, lists,
    dicts to be saved into the db as they are so i dont have to cobvet back
    and forth each time.
  • Denis McMahon at Nov 5, 2013 at 8:10 pm

    On Tue, 05 Nov 2013 20:09:42 +0200, Nick the Gr33k wrote:


    Denis, i have already provided my code trying to do what i need and i
    need some commendation on how to make it work.

    Nick, you're obviously trying to code way above your abilities.


    If you want me to write your code, you will have to pay my day rate, and
    you can't afford it. If you could afford it, you'd be paying someone
    competent already.


    Here, have a code example, this works, although the formatting might get
    broken. The database "ntg1" has a single table called str_idx defined as
    follows:


    create table idx_str ( idx int primary key, str varchar[1024] );


    #!/usr/bin/python


    import random
    import sqlite3


    def make_list( length ):
         l = []
         if ( length < 1 ):
             return l
         for count in range( 0, length ):
             s = '';
             for i in range( 1, random.randrange( 4, 12 ) ):
                 c = chr( random.randrange( 97, 123 ) )
                 s += c
             l.append( s )
         return l


    def list_to_string( l ):
         return "|".join( l )


    def string_to_list( s ):
         return s.split( "|" )


    l1 = make_list( 10 )
    print "l1 -> ", l1
    s = list_to_string( l1 )
    print "s -> ", s
    l2 = string_to_list( s )
    print "l2 -> ", l2
    print "l1 == l2 -> ", l2 == l1


    l2 = make_list( 10 )
    l3 = make_list( 10 )
    l4 = make_list( 10 )


    print "Original Lists"
    print "l1 -> ", l1
    print "l2 -> ", l2
    print "l3 -> ", l3
    print "l4 -> ", l4


    conn = sqlite3.connect( "ntg1" )
    cur = conn.cursor()
    cur.execute( "delete from idx_str where idx is not null" )
    cur.execute(
         "insert into idx_str values ( 1, '{0}' )".format(
             list_to_string( l1 ) ) )
    cur.execute(
         "insert into idx_str values ( 2, '{0}' )".format(
             list_to_string( l2 ) ) )
    cur.execute(
         "insert into idx_str values ( 3, '{0}' )".format(
             list_to_string( l3 ) ) )
    cur.execute(
         "insert into idx_str values ( 4, '{0}' )".format(
             list_to_string( l4 ) ) )
    conn.commit()
    conn.close()


    print "Lists now in DB"
    print "Reading 1 record at a time"


    conn2 = sqlite3.connect( "ntg1" )
    cur2 = conn2.cursor()


    cur2.execute( "select * from idx_str where idx = 1" );
    row = cur2.fetchone()
    print "stored 1 -> ", row[ 1 ]
    print "l1 -> ", string_to_list( row[ 1 ] )


    cur2.execute( "select * from idx_str where idx = 2" );
    row = cur2.fetchone()
    print "stored 2 -> ", row[ 1 ]
    print "l2 -> ", string_to_list( row[ 1 ] )


    cur2.execute( "select * from idx_str where idx = 3" );
    row = cur2.fetchone()
    print "stored 3 -> ", row[ 1 ]
    print "l3 -> ", string_to_list( row[ 1 ] )


    cur2.execute( "select * from idx_str where idx = 4" );
    row = cur2.fetchone()
    print "stored 4 -> ", row[ 1 ]
    print "l4 -> ", string_to_list( row[ 1 ] )


    conn2.close()


    print "Reading all records at once"


    conn3 = sqlite3.connect( "ntg1" )


    cur3 = conn3.cursor()


    cur3.execute( "select * from idx_str" );


    row = cur3.fetchone()


    while not row == None:


         print "stored ", row[ 0 ], " -> ", row[ 1 ]
         print "list ", row[ 0 ], " -> ", string_to_list( row[ 1 ] )
         row = cur3.fetchone()


    conn3.close()


    One thing you haven't considered, what happens if a user has so many
    downloads in his list that the converted list doesn't fit in the declared
    string column width in your database?


    --
    Denis McMahon, denismfmcmahon at gmail.com
  • Cameron Simpson at Nov 5, 2013 at 10:31 pm

    On 05Nov2013 20:09, Nikos wrote:
    O even better an rdbms than allows complex data such as tuples,
    lists, dicts to be saved into the db as they are so i dont have to
    cobvet back and forth each time.

    If you're just using the db for storage or adhoc and arbitrary
    python objects (and not querying the stored values via SQL - eg
    WHERE), then: make a column of type BLOB, convert Python values to
    bytes using pickle, store. And of course the reverse.


    It is not a great use of an RDB, but it seems to adhere to what you ask.
    --
    Cameron Simpson <cs@zip.com.au>
  • Bob gailer at Nov 5, 2013 at 11:15 pm
    There is also the shelve module.


    It uses pickle to marshal a Python object, then stores it in a file
    under a key.


    Sample code from the module documentation:


    import shelve


    d = shelve.open(filename) # open -- file may get suffix added by
    low-level library


    d[key] = data # store data at key (overwrites old data if using an
    existing key)
    data = d[key] # retrieve a COPY of data at key (raise KeyError if no
    such key)
    del d[key] # delete data stored at key (raises KeyError if no such key)
    flag = key in d # true if the key exists
    klist = list(d.keys()) # a list of all existing keys (slow!)




    --
    Bob Gailer
    919-636-4239
    Chapel Hill NC
  • Mark Lawrence at Nov 5, 2013 at 11:36 pm

    On 05/11/2013 22:31, Cameron Simpson wrote:
    On 05Nov2013 20:09, Nikos wrote:
    O even better an rdbms than allows complex data such as tuples,
    lists, dicts to be saved into the db as they are so i dont have to
    cobvet back and forth each time.
    If you're just using the db for storage or adhoc and arbitrary
    python objects (and not querying the stored values via SQL - eg
    WHERE), then: make a column of type BLOB, convert Python values to
    bytes using pickle, store. And of course the reverse.

    It is not a great use of an RDB, but it seems to adhere to what you ask.

    How do expect the OP to understand a BLOB or pickle or chutney when he
    doesn't understand why he can't iterate around a Nonetype object?


    --
    Python is the second best programming language in the world.
    But the best has yet to be invented. Christian Tismer


    Mark Lawrence
  • John Gordon at Nov 5, 2013 at 8:19 pm

    In <l5b8if$4k1$1 at dont-email.me> Nick the Gr33k <nikos.gr33k@gmail.com> writes:


    IAM STRUGGLING WITH IT 2 DAYS NOW AND I CANNOT GET IT TO WORK.
    ALL I WANT IT TO DO IS JUST
    1. RETRIEVE 3 COLUMNS THAT CONSIST OF 3 LONG STRINGS
    2. CONVERT LONG STRINGS TO LISTS
    3. ADD SOME CURRENT VALUES TO THOSE LISTS
    4. CONVERT FROM LISTS TO LONG STRINGS SO I CAN STORE SUCCESSFULLY LIST
    PYTHON DATATYPE TO MYSQL SCALAR STRING.
    EVERYHTIGN I TRIED FAILED.

    How did it fail?


    Error message?
    No results at all?
    Different results than you wanted? If so, how did they differ?


    --
    John Gordon Imagine what it must be like for a real medical doctor to
    gordon at panix.com watch 'House', or a real serial killer to watch 'Dexter'.
  • Mark Lawrence at Nov 5, 2013 at 9:17 pm

    On 05/11/2013 20:19, John Gordon wrote:
    In <l5b8if$4k1$1 at dont-email.me> Nick the Gr33k <nikos.gr33k@gmail.com> writes:
    IAM STRUGGLING WITH IT 2 DAYS NOW AND I CANNOT GET IT TO WORK.
    ALL I WANT IT TO DO IS JUST
    1. RETRIEVE 3 COLUMNS THAT CONSIST OF 3 LONG STRINGS
    2. CONVERT LONG STRINGS TO LISTS
    3. ADD SOME CURRENT VALUES TO THOSE LISTS
    4. CONVERT FROM LISTS TO LONG STRINGS SO I CAN STORE SUCCESSFULLY LIST
    PYTHON DATATYPE TO MYSQL SCALAR STRING.
    EVERYHTIGN I TRIED FAILED.
    How did it fail?

    Error message?
    No results at all?
    Different results than you wanted? If so, how did they differ?

    Dear John,


    This approach has already been tried by Dave Angel, Steven D'Aprano and
    Chris Angelico amongst others and has failed dismally so why should you
    be any different?


    I've taken a different approach. I've put the contract out to tender and
    hereby give you the winners
    http://www.mudefordwoodcommunitycentre.co.uk/playgroup-and-tiny-tots/


    Part of the rationale is that they're literally 100 yards from my front
    door so communicating with them effectively should be extremely easy.
    This will be particularly useful when it comes down to debugging such
    difficult issues as "> TypeError: 'NoneType' object is not iterable".


    I'm looking forward to working in partnership with them and am convinced
    that within a few days all of Nikos' problems will have been solved.


    --
    Python is the second best programming language in the world.
    But the best has yet to be invented. Christian Tismer


    Mark Lawrence
  • Mark Lawrence at Nov 5, 2013 at 9:42 pm

    On 05/11/2013 21:17, Mark Lawrence wrote:
    On 05/11/2013 20:19, John Gordon wrote:
    In <l5b8if$4k1$1 at dont-email.me> Nick the Gr33k <nikos.gr33k@gmail.com>
    writes:
    IAM STRUGGLING WITH IT 2 DAYS NOW AND I CANNOT GET IT TO WORK.
    ALL I WANT IT TO DO IS JUST
    1. RETRIEVE 3 COLUMNS THAT CONSIST OF 3 LONG STRINGS
    2. CONVERT LONG STRINGS TO LISTS
    3. ADD SOME CURRENT VALUES TO THOSE LISTS
    4. CONVERT FROM LISTS TO LONG STRINGS SO I CAN STORE SUCCESSFULLY LIST
    PYTHON DATATYPE TO MYSQL SCALAR STRING.
    EVERYHTIGN I TRIED FAILED.
    How did it fail?

    Error message?
    No results at all?
    Different results than you wanted? If so, how did they differ?
    Dear John,

    This approach has already been tried by Dave Angel, Steven D'Aprano and
    Chris Angelico amongst others and has failed dismally so why should you
    be any different?

    I've taken a different approach. I've put the contract out to tender and
    hereby give you the winners
    http://www.mudefordwoodcommunitycentre.co.uk/playgroup-and-tiny-tots/

    Part of the rationale is that they're literally 100 yards from my front
    door so communicating with them effectively should be extremely easy.
    This will be particularly useful when it comes down to debugging such
    difficult issues as "> TypeError: 'NoneType' object is not iterable".

    I'm looking forward to working in partnership with them and am convinced
    that within a few days all of Nikos' problems will have been solved.

    Oh dear it looks as if we might have to renegotiate the contract as I
    understand that we've an almost intractable problem to solve it's
    "AttributeError: 'NoneType' object has no attribute 'split'".


    --
    Python is the second best programming language in the world.
    But the best has yet to be invented. Christian Tismer


    Mark Lawrence
  • Chris Angelico at Nov 5, 2013 at 10:03 pm

    On Wed, Nov 6, 2013 at 8:17 AM, Mark Lawrence wrote:
    I've taken a different approach. I've put the contract out to tender and
    hereby give you the winners
    http://www.mudefordwoodcommunitycentre.co.uk/playgroup-and-tiny-tots/

    Sounds good! But I don't see a list of their technologies - do they
    use open source tools like Python, or is it proprietary stuff like
    Fisher-Price(tm) "My First Website"? The latter is probably sufficient
    for these tasks, but I would hope that your contractors are able to
    master real-world problems too.


    Nikos, once again you are getting yourself into a tizz over this,
    because the problem is urgent to you. You need to stop programming
    professionally and make it a hobby, so that you remove the time
    pressure. You cope badly with pressure, so why torture yourself? Get
    into a completely different industry for your day job, and do all your
    programming in the evenings. You'll find lots of things easier and
    less stressful.


    ChrisA
  • Nick the Gr33k at Nov 5, 2013 at 9:26 pm

    ???? 5/11/2013 10:19 ??, ?/? John Gordon ??????:
    In <l5b8if$4k1$1 at dont-email.me> Nick the Gr33k <nikos.gr33k@gmail.com> writes:
    IAM STRUGGLING WITH IT 2 DAYS NOW AND I CANNOT GET IT TO WORK.
    ALL I WANT IT TO DO IS JUST
    1. RETRIEVE 3 COLUMNS THAT CONSIST OF 3 LONG STRINGS
    2. CONVERT LONG STRINGS TO LISTS
    3. ADD SOME CURRENT VALUES TO THOSE LISTS
    4. CONVERT FROM LISTS TO LONG STRINGS SO I CAN STORE SUCCESSFULLY LIST
    PYTHON DATATYPE TO MYSQL SCALAR STRING.
    EVERYHTIGN I TRIED FAILED.
    How did it fail?

    Error message?
    No results at all?
    Different results than you wanted? If so, how did they differ?



    I know i'm close to solution, i can feel it but i have some issues.
    The code we arr discussing is the following:




    =================================================================================================================
    # ~ DATABASE INSERTS ~
    =================================================================================================================
    if cookieID != 'nikos' and ( os.path.exists( path + page ) or
    os.path.exists( cgi_path + page ) ) and re.search(
    r'(amazon|google|proxy|cloud|reverse|fetch|msn|who|spider|crawl|ping)',
    host ) is None:


      try:
       # if first time for webpage; create new record( primary key is
    automatic, hit is defaulted ), if page exists then update record
       cur.execute('''INSERT INTO counters (url) VALUES (%s) ON DUPLICATE KEY
    UPDATE hits = hits + 1''', page )
       cID = cur.lastrowid


       # fetch those columns that act as lists but are stored as strings
       cur.execute('''SELECT refs, visits, downloads FROM visitors WHERE
    counterID = %s and host = %s''', (cID, host) )
       data = cur.fetchone()


       ref = visit = download = []
       if cur.rowcount:
        # unpack data into variables
        (ref, visit, download) = data


        # retrieve long strings and convert them into lists respectively
        ref = ref.split()
        visit = visit.split()
        download = download.split()
       else:
        # initiate these values
        ref = ref
        visit = lastvisit
        download = ''


       refs = visits = downloads = []
       # add current values to each list respectively
       refs.append( ref )
       visits.append( visit )
       downloads.append( download )


       # convert lists back to longstrings
       refs = ', '.join( refs )
       visits = ', '.join( visits )
       downloads = ', '.join( downloads )


       # save this visit as an entry into database
       cur.execute('''INSERT INTO visitors (counterID, refs, host, city,
    useros, browser, visits, downloads) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)
           ON DUPLICATE KEY UPDATE refs = %s, visits = %s, hits = hits + 1,
    downloads = %s''',
           (cID, refs, host, city, useros, browser, visits, downloads, refs,
    visits, downloads) )


       con.commit()
      except pymysql.ProgrammingError as e:
       print( repr(e) )
       con.rollback()
       sys.exit(0)
    ===============================================




    [Tue Nov 05 23:21:52 2013] [error] [client 176.92.96.218] File
    "/home/nikos/public_html/cgi-bin/metrites.py", line 274, in <module>
    [Tue Nov 05 23:21:52 2013] [error] [client 176.92.96.218] visit =
    visit.split()
    [Tue Nov 05 23:21:52 2013] [error] [client 176.92.96.218]
    AttributeError: 'NoneType' object has no attribute 'split'
  • John Gordon at Nov 5, 2013 at 10:06 pm

    In <l5bnpc$71v$1 at dont-email.me> Nick the Gr33k <nikos.gr33k@gmail.com> writes:


    # fetch those columns that act as lists but are stored as strings
    cur.execute('''SELECT refs, visits, downloads FROM visitors WHERE
    counterID = %s and host = %s''', (cID, host) )
    data = cur.fetchone()
    if cur.rowcount:
    # unpack data into variables
    (ref, visit, download) = data

    visit = visit.split()

    [Tue Nov 05 23:21:52 2013] [error] [client 176.92.96.218] visit =
    visit.split()
    [Tue Nov 05 23:21:52 2013] [error] [client 176.92.96.218]
    AttributeError: 'NoneType' object has no attribute 'split'

    It appears that in the row you're fetching from the visitors table, the
    'visits' column is NULL.


    Your next step is track down why.


    --
    John Gordon Imagine what it must be like for a real medical doctor to
    gordon at panix.com watch 'House', or a real serial killer to watch 'Dexter'.
  • Nick the Gr33k at Nov 5, 2013 at 10:28 pm

    ???? 6/11/2013 12:06 ??, ?/? John Gordon ??????:
    In <l5bnpc$71v$1 at dont-email.me> Nick the Gr33k <nikos.gr33k@gmail.com> writes:
    # fetch those columns that act as lists but are stored as strings
    cur.execute('''SELECT refs, visits, downloads FROM visitors WHERE
    counterID = %s and host = %s''', (cID, host) )
    data = cur.fetchone()
    if cur.rowcount:
    # unpack data into variables
    (ref, visit, download) = data

    visit = visit.split()

    [Tue Nov 05 23:21:52 2013] [error] [client 176.92.96.218] visit =
    visit.split()
    [Tue Nov 05 23:21:52 2013] [error] [client 176.92.96.218]
    AttributeError: 'NoneType' object has no attribute 'split'
    It appears that in the row you're fetching from the visitors table, the
    'visits' column is NULL.

    Your next step is track down why.

    Indeed.


    The code i provided only worked once before it failed and managed to
    store this:




    counterID,host,refs,city,userOS,browser,visits,hits,download
    -------------------------------------------------------------
    1, 176-92-96-218.adsl.cyta.gr, Europe/Athens, Windows, Chrome,
    0000-00-00 00:00:00, 1, ''


    'visit's column is full of zeroes. Perhaps this is considered as Null
    for Python?


    But ref is also null, why the code didn't complain for 'ref' which is
    retrieved just before 'visits'?


    How, i eman what si the way to investigate this further?
  • Mark Lawrence at Nov 5, 2013 at 10:44 pm

    On 05/11/2013 22:28, Nick the Gr33k wrote:
    ???? 6/11/2013 12:06 ??, ?/? John Gordon ??????:
    In <l5bnpc$71v$1 at dont-email.me> Nick the Gr33k <nikos.gr33k@gmail.com>
    writes:
    # fetch those columns that act as lists but are stored as
    strings
    cur.execute('''SELECT refs, visits, downloads FROM visitors
    WHERE
    counterID = %s and host = %s''', (cID, host) )
    data = cur.fetchone()
    if cur.rowcount:
    # unpack data into variables
    (ref, visit, download) = data

    visit = visit.split()

    [Tue Nov 05 23:21:52 2013] [error] [client 176.92.96.218] visit =
    visit.split()
    [Tue Nov 05 23:21:52 2013] [error] [client 176.92.96.218]
    AttributeError: 'NoneType' object has no attribute 'split'
    It appears that in the row you're fetching from the visitors table, the
    'visits' column is NULL.

    Your next step is track down why.
    Indeed.

    The code i provided only worked once before it failed and managed to
    store this:


    counterID,host,refs,city,userOS,browser,visits,hits,download
    -------------------------------------------------------------
    1, 176-92-96-218.adsl.cyta.gr, Europe/Athens, Windows, Chrome,
    0000-00-00 00:00:00, 1, ''

    'visit's column is full of zeroes. Perhaps this is considered as Null
    for Python?

    But ref is also null, why the code didn't complain for 'ref' which is
    retrieved just before 'visits'?

    How, i eman what si the way to investigate this further?

    Don't worry about it. Just leave it with the crack team that I've put
    together for you. We should have some answers some time tomorrow.


    --
    Python is the second best programming language in the world.
    But the best has yet to be invented. Christian Tismer


    Mark Lawrence
  • John Gordon at Nov 5, 2013 at 10:54 pm

    In <l5brem$trk$1 at dont-email.me> Nick the Gr33k <nikos.gr33k@gmail.com> writes:


    The code i provided only worked once before it failed and managed to
    store this:

    counterID,host,refs,city,userOS,browser,visits,hits,download
    -------------------------------------------------------------
    1, 176-92-96-218.adsl.cyta.gr, Europe/Athens, Windows, Chrome,
    0000-00-00 00:00:00, 1, ''

    You've got a data inconsistancy here. You give nine column names, but
    only eight values. It's impossible to tell which value belongs to which
    column. Which value is missing?

    'visit's column is full of zeroes. Perhaps this is considered as Null
    for Python?

    Python does not consider zero to be a null value. (It will compare as
    equal to False, but that's not the same.)

    But ref is also null, why the code didn't complain for 'ref' which is
    retrieved just before 'visits'?

    Are you sure refs is null? According to the sample values you gave, refs
    is 'Europe/Athens'.


    --
    John Gordon Imagine what it must be like for a real medical doctor to
    gordon at panix.com watch 'House', or a real serial killer to watch 'Dexter'.
  • Nick the Gr33k at Nov 6, 2013 at 1:14 am

    ???? 6/11/2013 12:54 ??, ?/? John Gordon ??????:
    The code i provided only worked once before it failed and managed to
    store this:

    counterID,host,refs,city,userOS,browser,visits,hits,download
    -------------------------------------------------------------
    1, 176-92-96-218.adsl.cyta.gr, Europe/Athens, Windows, Chrome,
    0000-00-00 00:00:00, 1, ''
    You've got a data inconsistancy here. You give nine column names, but
    only eight values. It's impossible to tell which value belongs to which
    column. Which value is missing?



    Yes, you have a point but thats why because 'refs' was an empty string
    for some reason and i have should have made this clear by putting a ''
    like i did with the 'downloads' at the end.


    So it is not a matter of inconsistency.
    Its weird though why it was stored this way, refs and downloads that is
    as an empty strings.


    How, do i i proceed?
  • Mark Lawrence at Nov 6, 2013 at 8:49 am

    On 06/11/2013 01:14, Nick the Gr33k wrote:
    How, do i i proceed?

    If at first you don't succeed, keep asking on comp.lang.python until
    someone gives me the completely bodged solution that I keep asking for
    even if it's complete nonsense.


    --
    Python is the second best programming language in the world.
    But the best has yet to be invented. Christian Tismer


    Mark Lawrence
  • Joel Goldstick at Nov 6, 2013 at 1:57 pm

    On Wed, Nov 6, 2013 at 3:49 AM, Mark Lawrence wrote:
    On 06/11/2013 01:14, Nick the Gr33k wrote:


    How, do i i proceed?

    If at first you don't succeed, keep asking on comp.lang.python until someone
    gives me the completely bodged solution that I keep asking for even if it's
    complete nonsense.


    --
    Python is the second best programming language in the world.
    But the best has yet to be invented. Christian Tismer

    Mark Lawrence

    --
    https://mail.python.org/mailman/listinfo/python-list

    Nikos,


    I believe we are passed the time again where your enquiries slip from
    legitimate questions to trolling behavior. As was pointed out you
    aren't close to any solution. You should go off taking the advice and
    tutoring you have had here, do some reading (yes, actually learn and
    understand something), perhaps start from scratch reading basics of
    computer programming, databases, html, unicode, operating systems, web
    servers, etc. and come back when you have a solid basis to ask a
    question.


    I take my profession seriously. Its fun, but it takes a lot of
    reading and studying and practice to be good at it. The fact that you
    even presume that you are doing what software designers, or engineers
    or programmers do to ply the craft is disgusting. Its not that you
    don't know, its that you don't know and are an arrogant idiot to
    assume that you do.


    Your threads all end up being a very poor man's version of the movie
    Ground Hog Day. The fact that you don't recognize how rude and
    insulting you are, and how little you bring to the discussion will
    inevitably lead to more personal attacks since you aren't holding up
    your end of the technical back and forth.


    Your repeated behavior of starting new threads is just a boring
    childish method to demand attention. You have our attention. I
    believe that most of us if not all of us think you are the most
    aggravating, incompetent, rude, needy, manipulating, person we have
    ever seen online ever. Go away, and come back (months from now at
    least) when you have some redeeming human qualities.


    --
    Joel Goldstick
    http://joelgoldstick.com
  • Antoon Pardon at Nov 6, 2013 at 8:27 am

    Op 05-11-13 22:26, Nick the Gr33k schreef:


    I know i'm close to solution, i can feel it but i have some issues.
    The code we arr discussing is the following:

    No you are not. You are just doing random changes, without any
    understanding. If you had followed my suggestion and actually
    read the documentation of fetchone, you would have realised
    this wouldn't have worked either.


    Your resistance to reading the documentation is making you lose
    more time than you hoped to win by just going ahead and trying
    things blindly.

    [Tue Nov 05 23:21:52 2013] [error] [client 176.92.96.218] File
    "/home/nikos/public_html/cgi-bin/metrites.py", line 274, in <module>
    [Tue Nov 05 23:21:52 2013] [error] [client 176.92.96.218] visit =
    visit.split()
    [Tue Nov 05 23:21:52 2013] [error] [client 176.92.96.218]
    AttributeError: 'NoneType' object has no attribute 'split'

    This is essentially the same error you had before. You have an object
    of NoneType where you expect something different. Changing your code
    to use split instead of iteration won't change the fact that you have
    an object of NoneType.


    --
    Antoon Pardon
  • Piet van Oostrum at Nov 5, 2013 at 10:15 pm
    Nick the Gr33k <nikos.gr33k@gmail.com> writes:

    IAM STRUGGLING WITH IT 2 DAYS NOW AND I CANNOT GET IT TO WORK.

    ALL I WANT IT TO DO IS JUST

    1. RETRIEVE 3 COLUMNS THAT CONSIST OF 3 LONG STRINGS
    2. CONVERT LONG STRINGS TO LISTS
    3. ADD SOME CURRENT VALUES TO THOSE LISTS
    4. CONVERT FROM LISTS TO LONG STRINGS SO I CAN STORE SUCCESSFULLY LIST
    PYTHON DATATYPE TO MYSQL SCALAR STRING.

    EVERYHTIGN I TRIED FAILED.

    Then why don't you use the simple solution: use a relational database to store the data?
    --
    Piet van Oostrum <piet@vanoostrum.org>
    WWW: http://pietvanoostrum.com/
    PGP key: [8DAE142BE17999C4]
  • Nick the Gr33k at Nov 5, 2013 at 10:35 pm

    ???? 6/11/2013 12:15 ??, ?/? Piet van Oostrum ??????:
    Nick the Gr33k <nikos.gr33k@gmail.com> writes:
    IAM STRUGGLING WITH IT 2 DAYS NOW AND I CANNOT GET IT TO WORK.

    ALL I WANT IT TO DO IS JUST

    1. RETRIEVE 3 COLUMNS THAT CONSIST OF 3 LONG STRINGS
    2. CONVERT LONG STRINGS TO LISTS
    3. ADD SOME CURRENT VALUES TO THOSE LISTS
    4. CONVERT FROM LISTS TO LONG STRINGS SO I CAN STORE SUCCESSFULLY LIST
    PYTHON DATATYPE TO MYSQL SCALAR STRING.

    EVERYHTIGN I TRIED FAILED.
    Then why don't you use the simple solution: use a relational database to store the data?



    --
    How you mean?


    Up until now i was using for years MySQL.
    Now i realizes i just cannot store lists into it's columns because it
    does not support a collection datatype.


    perhaps its time to use postgresql so to avoid these coversions?


    does postgresql support "collection" columns?


    and is its synta
  • Denis McMahon at Nov 6, 2013 at 12:45 am

    On Wed, 06 Nov 2013 00:35:56 +0200, Nick the Gr33k wrote:


    Now i realizes i just cannot store lists into it's columns because it
    does not support a collection datatype.

    All databases support storing of collections, but *NOT THE WAY YOU WANT
    TO DO IT*


    You refuse to do it the proper way, so you have to bodge a solution ....
    oh wait .... I'm having a flashback here ....


    You have been told several times by several people how to do this
    properly. You insist on using your bodged up solution instead. OK, we'll
    all try and help you bodge up a solution, but I will tell you now, in
    advance, in very clear terms:


    *ONE DAY YOUR BODGED UP SOLUTION WILL BREAK BECAUSE THE TOTAL LIST OF
    TORRENTS THAT A USER HAS DOWNLOADED, WHEN PACKAGED INTO A STRING, WILL BE
    BIGGER THAN THE DATABASE STRING FIELD YOU ASSIGNED*


    There may be another issue that you need to consider.


    Supposing a torrent file name is 40 characters long. Supposing a user
    downloads up to 20 torrents a day. That means in one year, their list
    will contain 7300 entries, and the character string in the database will
    be 365 * 20 * 40 + 365 * 20 - 1 = 299299 characters.


    And you're going to convert this huge string into a list and then back
    into a string 20 times a day for every user just to add another 40
    characters to the end.


    Do you expect 100 users? 1,000? 10,000? 100,000?


    Let's assume that you have a million users. In a year, every day you're
    reading and writing about 6000 Gbytes a day from and to the database.


    Holy smoke batman, that's a lot of data.


    Or you could use the database properly, and just write the 40 byte
    torrent file name to the database with a pointer to the user's record
    every time a user downloads another torrent.


    --
    Denis McMahon, denismfmcmahon at gmail.com
  • Ben Finney at Nov 6, 2013 at 1:46 am

    Denis McMahon <denismfmcmahon@gmail.com> writes:


    You have been told several times by several people how to do this
    properly. You insist on using your bodged up solution instead. OK, we'll
    all try and help you bodge up a solution[?]

    Why? No-one here is obligated to help with implementing a solution we
    agree is bad.


    Now that helpful suggestions have been offered, and the OP continues to
    obstinately refuse to learn, why not leave it there? Why not leave this
    person to his own obstinacy and stop giving him the attention he
    demands?


    --
      \ ?Computers are useless. They can only give you answers.? ?Pablo |
       `\ Picasso |
    _o__) |
    Ben Finney
  • Grant Edwards at Nov 6, 2013 at 2:50 pm

    On 2013-11-06, Denis McMahon wrote:
    On Wed, 06 Nov 2013 00:35:56 +0200, Nick the Gr33k wrote:

    Now i realizes i just cannot store lists into it's columns because it
    does not support a collection datatype.
    All databases support storing of collections, but *NOT THE WAY YOU WANT
    TO DO IT*

    You refuse to do it the proper way, so you have to bodge a solution ....
    oh wait .... I'm having a flashback here ....

    To quote the great Casey Stengel "it's deja vu all over again".


    http://www.bostonbaseball.com/whitesox/baseball_extras/yogi.html


    --
    Grant Edwards grant.b.edwards Yow! I feel like a wet
                                       at parking meter on Darvon!
                                   gmail.com

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedNov 5, '13 at 5:06p
activeNov 6, '13 at 2:50p
posts32
users15
websitepython.org

People

Translate

site design / logo © 2022 Grokbase