FAQ
http://www.cvent.com/en/company/positions/database-architect.shtml

" When you think data, you chuckle at terabytes, relish petabytes, and dream
beyond exabytes. You think RDBMS may be the problem, rather than always
looking to it as the solution."

I have seen that rdbms is not the perfect solution for everything. For
example, google has their own database. Verisign has a database they wrote
in C that does billions of executions/day. I read that facebook is using
some kind of graphing database.

However, when I work with internet developers (it is almost always java guys
who do this), they tend to run their mouths, but have never worked on
anything at all and deliver garbage. The java guys at Fannie Mae tried to
ignore the database for their mortgage backed security application. They
spent $600 million (this number came from their executives) and they ended
up throwing it all away.

Anyone have experience with people who actually know anything when they talk
about this type of thing? here is a short note, I found about a database
product that my space uses.

http://www.cio.com/article/361313/Startup_Launches_Internet_Scale_Database

Search Discussions

  • Robert Freeman at May 13, 2011 at 2:46 am
    I have the quote, but not the source of the quote here that I like that I think
    addresses this thinking...

    "Anyone can re-invent every system out there from the word go..... is it worth
    it? In most cases, of course not! Why? Because they are increasing the total
    cost of development when they ignored previous IT science and development. They
    have ignored what is already there, ready to be used, debugged, tuned and
    optimized ten orders of magnitude better than what they would be able to achieve
    in a typically costed product.....

    ....These folks should try to use te software that is already out there instead
    of re-inventing the wheel for every project they get involved in."

    This quote was more directed at the j2ee crowd that wants to ignore the use of
    FK's and other constraints, but I think it applies here. How much does this
    "architect" think Google spent on their database solution (and by the way, they
    still use Oracle). These home grown database solutions are expensive, and very
    few companies really have the resources or need to build such solutions. Besides
    being expensive, custom made solutions are often built to deal with custom made
    problems. As a result they work great for one kind of computing but suck at
    other kinds of computing. Take certain data warehouse solutions. They are great
    when concurrency is low but you throw any kind of concurency or transactional
    load at them and they fall apart.... while on the other hand Oracle can support
    the load (if properly architected) and then some.

    The problem I see with developers is not that they run their mouths off, but
    that they don't ask questions, and don't bother to understand the product they
    are using (the same might be said about some DBA's too). They don't bother to
    understand that they have a tool at their hand, and it works efficiently when
    used in the way intended. All to often, they don't bother to use it the way it's
    intended. They want to say "it should work this way", "it should be able to do
    this", or better yet, "I don't trust the database to do what I want."

    Then they use tools like hibernate that inherently make for in-efficient
    interfacing with the database, use procedural processing instead of group/set
    processing and don't write tuned SQL code and have the gaul to wonder, why is
    the database so slow.

    Sigh..... I love developers, I really do..... but they need to be taught.....
    they need to learn. That is the point in some senses of my DBA 3.0 series on my
    blog (which I need to post part three on).... that we as DBA's are somewhat
    responsible for this. We have not been evangelests, we have not sold our wares,
    or taught the lessons that need to be taught.

    Have we really been good stewards of our knowledge? That is the question.

    Robert

    Robert G. Freeman
    Master Principal Consultant, Oracle Corporation, Oracle ACE
    Author of various books on RMAN, New Features and this shorter signature line.
    Blog: http://robertgfreeman.blogspot.com

    Note: THIS EMAIL IS NOT AN OFFICIAL ORACLE SUPPORT COMMUNICATION. It is just the
    opinion of one Oracle employee. I can be wrong, have been wrong in the past and
    will be wrong in the future. If your problem is a critical production problem,
    you should always contact Oracle support for assistance. Statements in this
    email in no way represent Oracle Corporation or any subsidiaries and reflect
    only the opinion of the author of this email.

    From: Dba DBA
    To: ORACLE-L
    Sent: Thu, May 12, 2011 6:07:37 PM
    Subject: company database architect that doesn't like rdbms?

    http://www.cvent.com/en/company/positions/database-architect.shtml

    " When you think data, you chuckle at terabytes, relish petabytes, and dream
    beyond exabytes. You think RDBMS may be the problem, rather than always looking
    to it as the solution."

    I have seen that rdbms is not the perfect solution for everything. For example,
    google has their own database. Verisign has a database they wrote in C that does
    billions of executions/day. I read that facebook is using some kind of graphing
    database.

    However, when I work with internet developers (it is almost always java guys who
    do this), they tend to run their mouths, but have never worked on anything at
    all and deliver garbage. The java guys at Fannie Mae tried to ignore the
    database for their mortgage backed security application. They spent $600 million
    (this number came from their executives) and they ended up throwing it all
    away.

    Anyone have experience with people who actually know anything when they talk
    about this type of thing? here is a short note, I found about a database product
    that my space uses.

    http://www.cio.com/article/361313/Startup_Launches_Internet_Scale_Database
  • Stephane Faroult at May 13, 2011 at 7:35 am
    Robert,
    Then they use tools like hibernate that inherently make for
    in-efficient interfacing with the database, use procedural processing
    instead of group/set processing and don't write tuned SQL code and
    have the gaul to wonder, why is the database so slow.

    Sigh..... I love developers, I really do..... but they need to be
    taught..... they need to learn. That is the point in some senses of my
    DBA 3.0 series on my blog (which I need to post part three on)....
    that we as DBA's are somewhat responsible for this. We have not been
    evangelests, we have not sold our wares, or taught the lessons that
    need to be taught.

    Have we really been good stewards of our knowledge? That is the question.
    While I agree with the diagnostic, and as someone who has been
    doing, and is still doing, like yourself some effort to educate the
    masses (if you search for "oracle" on Youtube you'll usually find me
    ahead of your employer), I am not sure that grass-root-level guilt is
    the way to go. For one thing, I'd lay squarely a large part of the blame
    on vendors themselves and the marketing discourse, "the database will
    fix it automagically" approach, the supposedly bright ideas to do it
    that, in the long term, proved to be not as bright as intended
    (CURSOR_SHARING=FORCE, anyone? But of course we can't review the code
    and use bind variables where needed). While acknowledging that a handful
    of people, the most visible of whom is probably Tom Kyte, have always
    advocated sound coding practices over the use of presumably clever
    features, I am not sure that this is the discourse that CIOs have heard.
    I'd rather think that they have heard that any idiot can "persist" data
    in a database, and that if they spend enough bucks on Exadata and the
    like, "their" applications will fly even with cheap outsourced coders.
    "You won't need to touch the code". Yes, and that's precisely the
    problem. One day, the code will hit the fan anyway.

    The problem is very deep. It starts at the university. I have seen on
    a French forum a student talking about what his DBMS professor had said,
    which was hotly debated; there was not enough context for me to decide
    whether what the professor had said was wrong or not. Anyway, the
    student defended the lecturer by saying that he was the CIO at the main
    site of a major French corporation (which he named and which is known
    all over the world). I presume that this man's latest hands-on
    experience of a DBMS dates back to the days when he started in the trade
    as a young IMS/DL1 developer. The problem isn't only French. I've taught
    in a Canadian University a long time ago, some CS professors only logged
    into systems to enter grades. In some of the database videos posted in
    the internet by the prestigious Indian Institutes of Technology the
    lecturer explains subqueries with the example of a correlated IN () (uh?).
    I dare not imagine what is taught in lesser educational institutes.

    There is also a growing disconnect between infrastructure, to which DBAs
    are usually attached, and development, made even worse by the
    cloudification. Although both are turning in their own ways to a
    point-and-click affair, if you show the OEM graphs to a developer he or
    she will not be able to make much sense of them (and I am often led
    myself to wonder what kind of action they are supposed to trigger).
    Conversely, most DBAs are not competent programmers - and I have no
    problem with this, it's a different job. If job ads always refer to "SQL
    tuning" in the requirement list for DBAs, it usually means "spotting the
    missing index or unused one" and not much else. My own vision of SQL
    "tuning" is usually rewriting queries, and sometimes a whole process,
    from scratch after understanding the initial intent, but I don't
    consider it a part of a DBA job. And the bigger the company, the greater
    the disconnect.
    If some of us still have on their shelves, and still open from times to
    times, "The Art of Computer Programming", we are in a minority - and
    when I say "we" I can include DBAs (normal) and developers (ouch). I no
    longer count the number of crisis-meetings-to-fix-performance-issues
    with DBAs saying that the number of buffer busy latches was much too
    high to developers who have trouble understanding what a pointer is
    (only 40% of programmers understand pointers according to Joel Spolsky).
    Wasted time. I have never seen anything more fruitful than sitting next
    to a programmer, recoding part of the program and showing the difference.

    I agree with you that any little thing a DBA can do to explain how
    things work to a developer helps. But it's just a drop in the ocean.
  • Michael Dinh at May 13, 2011 at 1:07 pm
    At the risk of being black listed, we all know what the problems are.

    What's the solution???

    How did the organization get here, how do they move away, and what will it take?

    Is it at the point of no return to live with and deal with?

    From: oracle-l-bounce_at_freelists.org [oracle-l-bounce_at_freelists.org] On Behalf Of Stephane Faroult [sfaroult_at_roughsea.com]
    Sent: Friday, May 13, 2011 12:35 AM
    To: robertgfreeman_at_yahoo.com
    Cc: oracledbaquestions_at_gmail.com; ORACLE-L
    Subject: Re: company database architect that doesn't like rdbms?

    Robert,
    Then they use tools like hibernate that inherently make for in-efficient interfacing with the database, use procedural processing instead of group/set processing and don't write tuned SQL code and have the gaul to wonder, why is the database so slow.

    Sigh..... I love developers, I really do..... but they need to be taught..... they need to learn. That is the point in some senses of my DBA 3.0 series on my blog (which I need to post part three on).... that we as DBA's are somewhat responsible for this. We have not been evangelests, we have not sold our wares, or taught the lessons that need to be taught.

    Have we really been good stewards of our knowledge? That is the question.

    While I agree with the diagnostic, and as someone who has been doing, and is still doing, like yourself some effort to educate the masses (if you search for "oracle" on Youtube you'll usually find me ahead of your employer), I am not sure that grass-root-level guilt is the way to go. For one thing, I'd lay squarely a large part of the blame on vendors themselves and the marketing discourse, "the database will fix it automagically" approach, the supposedly bright ideas to do it that, in the long term, proved to be not as bright as intended (CURSOR_SHARING=FORCE, anyone? But of course we can't review the code and use bind variables where needed). While acknowledging that a handful of people, the most visible of whom is probably Tom Kyte, have always advocated sound coding practices over the use of presumably clever features, I am not sure that this is the discourse that CIOs have heard. I'd rather think that they have heard that any idiot can "persist" data in a database, an
    d that if they spend enough bucks on Exadata and the like, "their" applications will fly even with cheap outsourced coders. "You won't need to touch the code". Yes, and that's precisely the problem. One day, the code will hit the fan anyway.

    The problem is very deep. It starts at the university. I have seen on a French forum a student talking about what his DBMS professor had said, which was hotly debated; there was not enough context for me to decide whether what the professor had said was wrong or not. Anyway, the student defended the lecturer by saying that he was the CIO at the main site of a major French corporation (which he named and which is known all over the world). I presume that this man's latest hands-on experience of a DBMS dates back to the days when he started in the trade as a young IMS/DL1 developer. The problem isn't only French. I've taught in a Canadian University a long time ago, some CS professors only logged into systems to enter grades. In some of the database videos posted in the internet by the prestigious Indian Institutes of Technology the lecturer explains subqueries with the example of a correlated IN () (uh?).
    I dare not imagine what is taught in lesser educational institutes.

    There is also a growing disconnect between infrastructure, to which DBAs are usually attached, and development, made even worse by the cloudification. Although both are turning in their own ways to a point-and-click affair, if you show the OEM graphs to a developer he or she will not be able to make much sense of them (and I am often led myself to wonder what kind of action they are supposed to trigger). Conversely, most DBAs are not competent programmers - and I have no problem with this, it's a different job. If job ads always refer to "SQL tuning" in the requirement list for DBAs, it usually means "spotting the missing index or unused one" and not much else. My own vision of SQL "tuning" is usually rewriting queries, and sometimes a whole process, from scratch after understanding the initial intent, but I don't consider it a part of a DBA job. And the bigger the company, the greater the disconnect.
    If some of us still have on their shelves, and still open from times to times, "The Art of Computer Programming", we are in a minority - and when I say "we" I can include DBAs (normal) and developers (ouch). I no longer count the number of crisis-meetings-to-fix-performance-issues with DBAs saying that the number of buffer busy latches was much too high to developers who have trouble understanding what a pointer is (only 40% of programmers understand pointers according to Joel Spolsky). Wasted time. I have never seen anything more fruitful than sitting next to a programmer, recoding part of the program and showing the difference.

    I agree with you that any little thing a DBA can do to explain how things work to a developer helps. But it's just a drop in the ocean.
  • Anonymous at May 13, 2011 at 7:36 am
    Morning Robert,

    I'm not the best DBA in the world by a long shot, but I'm not bad, and I
    was (and still am) a developer, so I speak from both sides of the fence.
    ....These folks should try to use the software that is
    already out there instead of re-inventing the wheel for
    every project they get involved in."
    Agree, 100%.
    This quote was more directed at the j2ee
    Agree 100%. In fact anyone thinking of using Java - in my opinion -
    should be kept well away from any database!
    crowd that wants to
    ignore the use of FK's and other constraints, but I think it
    applies here.
    Please, don't start me off on this subject. I have lost count of the
    number of "discussions" I've had with various vendors and developers who
    think that the application is the only place in the entire multi-verse
    that will access the data in the database. They simply don't "get it"
    that the data is the most important thing, not their application which
    may evolve and will go extinct long before the data is no longer
    required.

    I usually find - by cunning use of a quick SQL statement to update a
    table - that the application is written this way in order to assume
    that the data it reads from the database will be consistent with what
    the application wrote into the database. I've broken numerous
    applications simply by setting a column to a "this shouldn't happen"
    value and seeing what happens. Hugely enjoyable!;-)

    Management sometimes don't help either. A system I inherited a number of
    years back required "create any procedure" and "execute any procedure"
    to work "properly". I demonstrated to a manager that this was a huge
    security hole by creating SYSTEM.DROP_USER procedure and when executed,
    dropped the entire application schema, all the data and everything. His
    response? Don't tell the customers about this. Sigh.
    Besides being expensive,
    custom made solutions are often built to deal with custom
    made problems.
    And say what you like about Oracle Support, but with these home brewed
    systems, what do you do for support when the chief "architect" and
    his/her followers decide to leave the company?
    The problem I see with developers is not that they run their
    mouths off, but that they don't ask questions, and don't
    bother to understand the product they are using (the same
    might be said about some DBA's too).
    I agree again. On both counts. We have, at present, a set of standards
    that we supply to all our developers and to any vendor wishing to write
    for us. Rules 1 and two are, slightly paraphrased:

    Know your database. Know how it works, what features it gives you and
    how use them. We have paid an expensive license to use these features so
    your application should use them where appropriate.

    Know your development tool. Know its features and what it allows you
    to do. Make good use of it.

    Actually, rules three & four are good as well:

    3. Instrument your code. You will appreciate it when a bug arises and
    you have information to help track it down quickly.

    4. There is no such thing as a "database agnostic" system. Code for
    Oracle won't work on DB2. Anything written in this manner must be using
    the minimum level of features common across all potential databases -
    see rule 1 above re licensing costs and features!
    They want to say "it should work this way", "it should be able to do
    this", or better yet, "I don't trust the database to do
    what I want."
    Welcome to my world! And, I have noticed, it's always the Java
    developers who re-invent the wheel. What is it that makes them do this?
    I have a feeling that they just want to "show off" their "skills" in
    writing lots and lots of Java code. Don't they know that the database
    server is running oodles more power than their middle tier server and
    can do the work faster, better and cheaper (in power terms) than
    anything that their pseudo-compiled code can do?

    That was a hypothetical question by the way, they usually don't know!

    I once argued about Java performance with a very good Java developer. My
    point was that performance basically sucked - compare the old Oracle 8.0
    installer written in compiled C with the new Java installer. He said
    that he had implemented a system, in Java, that handled 5,000
    transactions an hour (Fast eh?) and that performance was excellent.
    Then, he admitted, that to get it up to that "speed" they had to assume
    that no errors would occur and took all the error checking code out!

    More recently, in a place I worked, a system was created in Java to
    replace an old creaking system. It was predicted that it would execute
    40,000 transactions per hour. Then 30,000, then 20,000, then 10,000 then
    they stopped predicting and went very quiet. It finally went in to
    production and ran alongside the old system for about three years. They
    never did switch off the old system, and now, the old system is still
    running while the "new improved" replacement has been decommissioned. A
    complete waste of time, effort and money.

    My point being that the old system took advantage of the features of the
    database and used them to good effect. The new system didn't and simply
    treated the database as a "bit-bucket".
    Then they use tools like hibernate that inherently make for
    in-efficient interfacing with the database, use procedural
    processing instead of group/set processing and don't write
    tuned SQL code
    I thoroughly recommend Stephane Faroult and Pascal L'Hermite's excellent
    book on refactoring - Refactoring SQL Applications, available from all
    good amazon.co.uk shops here
    http://www.amazon.co.uk/Refactoring-SQL-Applications-Stephane-Faroult/dp
    /0596514972/ref=sr_1_17?s=books&ie=UTF8&qid=1305270242&sr=1-17 (hope the
    link doesn't wrap!). I think thatif you replace ".co.uk" with ".com" in
    the above URL, it will work fine. That's what I normally do (the other
    way around) when someone posts an amazon.com URL.
    and have the gaul to wonder, why is the database so slow.
    Because there are *never* any problems with developer code, the problems
    are *always* with the database. This is what we DBAs have to deal with
    frequently - it's not working, it *must* be the database. I usually
    reply - give me the evidence. There is none. I supply them with a 10046
    trace at level 12 (just to be mean!) and prove that the application is
    broken. Not many developers can actually read and interpret a 10046
    trace, regardless of my best efforts to teach them (I could be a bad
    teacher of course!).

    Also, getting back to the features of the database, we have tkprof
    (although I use other tools, my favourite being the perl script from
    Norbert Debes excellent book Secrets of the Oracle Database -
    http://www.amazon.co.uk/Secrets-Oracle-Database-Experts-Voice/dp/1430219
    521/ref=sr_1_1?ie=UTF8&s=books&qid=1305270500&sr=1-1 (also available on
    Kindle) which basically does a "Method R" on the trace file.

    Ever heard of a developer using DBMS_PROFILER?
    Sigh..... I love developers, I really do..... but they need
    to be taught..... they need to learn.
    But they need to listen and understand as well. This is what I find is
    wrong, programming "flavour of the month" paradigms come and go, why
    bother to deeply learn and understand something when you know it will be
    replaced "very soon now" by something new - like "VB.NET on Rails" or
    similar!

    (Ok I was kidding about VB.NET on Rails!)
    That is the point in
    some senses of my DBA 3.0 series on my blog (which I need to
    post part three on).... that we as DBA's are somewhat
    responsible for this. We have not been evangelists, we have
    not sold our wares, or taught the lessons that need to be taught.
    I disagree. I'm getting to the stage of living through "Groundhog Day"
    explaining to people why a referential integrity constraint is a good
    thing as opposed to having the application check it for me and so on.
    They seem to have, in most cases, a mental block or a downright smugness
    that makes them think that a DBA doesn't know anything about programming
    applications.

    In a past life I wrote code and every Monday we had a team meeting to go
    through someone's work of the last week. All the developers had a say in
    what was "wrong" what could have been done to improve it and so on. Any
    deviation from standards was jumped on and so forth. Peer review in
    other words. I don't know of any institution where that takes place
    anymore - there may be some - but nowadays it's all just write it and
    bung it into production if it looks like it runs ok. Also, people
    nowadays get very possessive about their code, thou shalt not criticise
    lest the wrath of a disgruntled developer befall you.

    And, as for so called vendors, I still get instructions to :

    create user app_owner identified by app_owner ...;
    grant connect resource dba to app_owner;
    grant create session to app_owner;

    And then, the application and everyone using it logs in as app_owner.
    Did I mention that the password is hard coded into the application? Do
    they realise exactly what they are doing? How about connect & resource
    being changed between Oracle versions and not meaning the same as they
    did back in the good old days of 8i and 9i?

    I despair, and also refuse to allow any such rubbish near my (yes, my!)
    databases.
    Have we really been good stewards of our knowledge? That is
    the question.
    I think, in many cases, that we have been good stewards and teachers.
    And in many cases, we have been completely wasting our time. :-(

    "Those who can, do. Those who can't write Java code.";-)

    Cheers,
    Norm.

    PS. Usual rules apply, the above are purely my opinions, observations
    and facts. They are not necessarily those of my employer (at the
    moment!) whose disclaimer will be almost as long as my eRant above.
    Also, other opinions are available - but mostly, they will be wrong!;-)

    Information in this message may be confidential and may be legally privileged. If you have received this message by mistake, please notify the sender immediately, delete it and do not copy it to anyone else.

    We have checked this email and its attachments for viruses. But you should still check any attachment before opening it.
    We may have to make this message and any reply to it public if asked to under the Freedom of Information Act, Data Protection Act or for litigation. Email messages and attachments sent to or from any Environment Agency address may also be accessed by someone other than the sender or recipient, for business purposes.

    If we have sent you information and you wish to use it please read our terms and conditions which you can get by calling us on 08708 506 506. Find out more about the Environment Agency at www.environment-agency.gov.uk
  • Goulet, Richard at May 13, 2011 at 2:23 pm
    Well, I'm afraid that I have to seriously agree with Robert on this.
    The desire to "reinvent the wheel" all the time in pursuit of a better
    mousetrap has been one of the downfalls of the commercial software
    market. For some reason the open source community has done better,
    though not perfect. Part of that I believe is human ego and the NIH
    syndrome (Not Invented Here). A lot of this reinvention is pointed
    towards making things faster or better by eliminating "non essential"
    activities. Come to think of it we have a proponent of that somewhere
    around here, you remember "The fastest way to do anything is not to.".
    Problem with that is that many times that non-essential activity is what
    lets you know what went wrong when it does. As for developers, they
    want to believe that thy know everything (here comes that ego again).
    I've many who want dba privileges and access to system and sys whenever
    they "need" it. On the other hand I've found a large pile of local
    developers who are starved for information, help, and guidance. Try
    feeding them, careful now you may get what you're asking for.



    Richard Goulet

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouporacle-l @
categoriesoracle
postedMay 13, '11 at 12:07a
activeMay 13, '11 at 2:23p
posts6
users6
websiteoracle.com

People

Translate

site design / logo © 2022 Grokbase