Grokbase
Topics Posts Groups | in
x
[ help ]

Just Someone (just...@gmail.com)

Profile | Posts (25)Page 1 of 2: 1 2 > >>
1) Just Someone Re: Schema search_path and views
| +1 vote
Cool! That explains it fully. So i guess there will be a better performance to the pre-generated...
PostgreSQL - General
[ Profile | Reply to group ] [ Flat  Thread  Threaded ]
Cool!

That explains it fully. So i guess there will be a better performance
to the pre-generated views at the price of more views.

Thanks!


On 11/6/06, Merlin Moncure <mmoncure@gmail.com> wrote:
> On 11/6/06, Just Someone <just.some@gmail.com> wrote:
> > I have a database with multiple schemas all with the same structure
> > (but of course different data...).
> >
> > I want to create a view that will be created in a shared schema, and
> > when executed will be executed against the current schema. Whenever I
> > try it, it seems the view is linked to a specific schema used when
> > creating it, and doesn't reevaluates based on the current schema.
>
> no, or not exactly. views resolve the search path when they are
> generated. this is a fundemental part of how they work. functions,
> however, are a bit different. the plans are lazily generated and
> 'stick' to the tables that are resolved in the search path when the
> plan is generated, which is basically the first time you run them in a
> session. so, you could in theory do what you want with a view if it
> called functions for all the switchable parts.
>
> merlin
>


--
Family management on rails: http://www.famundo.com - coming soon!
My development related blog: http://devblog.famundo.com
2) Just Someone Schema search_path and views
| +1 vote
I have a database with multiple schemas all with the same structure I want to create a view that...
PostgreSQL - General
[ Profile | Reply to group ] [ Flat  Thread  Threaded ]
I have a database with multiple schemas all with the same structure
(but of course different data...).

I want to create a view that will be created in a shared schema, and
when executed will be executed against the current schema. Whenever I
try it, it seems the view is linked to a specific schema used when
creating it, and doesn't reevaluates based on the current schema.

Here is the pseudo structure/code:

schema1:
=======
create table t1 ...

schema2:
=======
create table t1 ...

shared_schema:
============
create table t3 ...

create the view:
===========
set search_path to shared_schema, schema1;
create view view1 as select * from t1;

try the view:
========
set search_path to shared_schema, schema1;
select * from view1;
set search_path to shared_schema, schema2;
select * from view1;

Results:
======
In the above, both select * from view1; will return the same data,
though the search path changed.

Is there a way to make the view use the current search_path?



--
Family management on rails: http://www.famundo.com - coming soon!
My development related blog: http://devblog.famundo.com
3) Just Someone Re: using schema's for data separation
| +1 vote
I am using a similar solution, and I tested it with a test containing 20K+ different schemas....
PostgreSQL - General
[ Profile | Reply to group ] [ Flat  Thread  Threaded ]
I am using a similar solution, and I tested it with a test containing
20K+ different schemas. Postgres didn't show slowness at all even
after the 20K (over 2 million total tables) were created. So I have
feeling it can grow even more.

Guy.



On 9/28/06, snacktime <snacktime@gmail.com> wrote:
> I'm re evaluating a few design choices I made a while back, and one
> that keeps coming to the forefront is data separation. We store
> sensitive information for clients. A database for each client isn't
> really workable, or at least I've never though of a way to make it
> workable, as we have several thousand clients and the databases all
> have to be accessed through a limited number of web applications where
> performance is important and things like persistant connections are a
> must. I've always been paranoid about a programmer error in an
> application resulting in data from multiple clients getting mixed
> together. Right now we create a schema for each client, with each
> schema having the same tables. The connections to the database are
> from an unprivileged user, and everything goes through functions that
> run at the necessary privileges. We us set_search_path to
> public,user. User data is in schema user and the functions are in the
> public schema. Every table has a client_id column.
>
> This has worked well so far but it's a real pain to manage and as we
> ramp up I'm not sure it's going to scale that well. So anyways my
> questions is this. Am I being too paranoid about putting all the data
> into one set of tables in a common schema? For thousands of clients
> what would you do?
>
> Chris
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
> http://archives.postgresql.org
>


--
Family management on rails: http://www.famundo.com - coming soon!
My development related blog: http://devblog.famundo.com
4) Just Someone Re: SELinux + CREATE TABLESPACE = ?
| +1 vote
If you rather keep SELinux on, you can still set the SELinux context on the directory where you...
PostgreSQL - General
[ Profile | Reply to group ] [ Flat  Thread  Threaded ]
If you rather keep SELinux on, you can still set the SELinux context
on the directory where you want the tablespaces to one postgres will
like.

To find what is the permissions you need, you can use ls -Z. It will
list the SELinux context. Check /var/lib/pgsql/data (or wherever
postgres data is pointing to), and then set this same permission on
the target dir using chcon.

For example, on my FC4 system all subdirectories on the data directory have:
root:object_r:postgresql_db_t or user_u:object_r:postgresql_db_t

So if you want to chage /path/to/foo/which/is/not/under/pgdata, run
(as root or sudo):

chcon root:object_r:postgresql_db_t /path/to/foo/which/is/not/under/pgdata

This way postgres can access it, and you get the SELinux security.

Bye,

Guy.

http://www.famundo.com
http://devblog.famundo.com
- Hide quoted text -


On 8/2/06, David Fetter <david@fetter.org> wrote:
> On Wed, Aug 02, 2006 at 04:47:36PM -0700, David Fetter wrote:
> > Folks,
> >
> > This may have come up before, but I didn't see a specific answer in
> > the archives.
> >
> > When I try to do:
> >
> > CREATE TABLESPACE foo LOCATION '/path/to/foo/which/is/not/under/$PGDATA';
> >
> > I get:
> >
> > ERROR: could not set permissions on directory "/path/to/foo/which/is/not/under/$PGDATA"
> >
> > Apparently this is a SELinux problem. How do I set the policy to
> > allow for this, or if that's not possible, how do I disable SELinux?
> >
> > Thanks in advance :)
>
> Pardon my self-followup for the archives :)
>
>
> Thanks to Talha Khan, who said:
>
> >    setenforce 1;
> >
> >    will disable SELINUX
>
> Thanks also to Clodoaldo Pinto, who said:
>
> > >Apparently this is a SELinux problem.
> >
> > Confirm it looking for a message in /var/log/messages.
> >
> > >How do I set the policy to allow for this,
> >
> > This Fedora FAQ is good:
> > http://fedora.redhat.com/docs/selinux-faq-fc5/#faq-div-controlling-selinux
> >
> > >or if that's not possible, how do I disable SELinux?
> >
> > edit /ect/selinux/config
>
> Cheers,
> D
> --
> David Fetter <david@fetter.org> http://fetter.org/
> phone: +1 415 235 3778        AIM: dfetter666
>                               Skype: davidfetter
>
> Remember to vote!
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faq
>
5) Just Someone Re: A better AND query?
| +1 vote
The schema can change, but I rather not. The use case is a web app where you can tag items with...
PostgreSQL - General
[ Profile | Reply to group ] [ Flat  Thread  Threaded ]
The schema can change, but I rather not.

The use case is a web app where you can tag items with tags
(many-2-many). There are multiple items you can tag: contacts,
schedules, lists, etc... And then you can search and categorize by
tags. The standard for this if you look aroung the web is to retrieve
the tagged records with any of the tags you select. Effectively an OR
query.

What I'm trying to do is search for items matching multiple tags at
the same time - and AND query. So that I can bring up all contacts
that are tagged with friends and movie-lovers.

Hope that clears it up a bit...

Guy.


On 5/9/06, Wayne Conrad <wconrad@yagni.com> wrote:
> > tagged_type int -- points to the table this tag is tagging
>
> My head exploded right about here. Is the schema written in stone, or
> can it change?
>
> What is the use case for this schema? What's it for? What is a "tag"
> about?
>
> Best Regards,
>         Wayne Conrad
>


--
Family management on rails: http://www.famundo.com - coming soon!
My development related blog: http://devblog.famundo.com
6) Just Someone A better AND query?
paperclip | +1 vote
I'm trying to generate a query that will handle tags matching in a database. The simplified...
PostgreSQL - General
[ Profile | Reply to group ] [ Flat  Thread  Threaded ]
I'm trying to generate a query that will handle tags matching in a database.

The simplified structure is

create table contacts (
  id  serial primary key,
  name varchar
);

create table books (
  id  serial primary key,
  name varchar
);

create table tags (
  id serial primary key,
  name varchar
);

create table taggings (
  tag_id int,
  tagged_id int,
  tagged_type int -- points to the table this tag is tagging
);

What I want to now achieve is to find all items that are tagged with
the same set of tags. So it's an AND matching on a list of tags I
have.

I have two types of matching. One is within the same object type
(where both tagged objects are the same, say two books with the same
set of tags) and one that will find ANY object that's tagged with the
same tag (like book and contact)

Current query (for the same object type) I am using is the following,
for a list of 4 tags called summer, winter, spring and fall.

SELECT *
FROM contacts WHERE 4 = ( SELECT COUNT(*)

    FROM tags, taggings
WHERE tags.id = taggings.tag_id

          AND lower(tags.name) IN ( 'summer' , 'winter', 'spring', 'fall' )

          AND taggings.tagged_type = 1
          AND taggings.tagged_id = contacts.id);


The query to match all the objects tagged with a given set of tags is:

SELECT DISTINCT taggings.tagged_id, taggings.tagged_type
FROM taggings WHERE 4 = ( SELECT COUNT(*)
    FROM tags, taggings as taggings2 WHERE tags.id = taggings2.tag_id
      AND lower(tags.name) IN ( 'summer' , 'winter', 'spring', 'fall' )
      AND taggings.tagged_type = taggings2.tagged_type
      AND taggings.tagged_id = taggings2.tagged_id );

The idea in both is to see that I find the number of tags needed.

I've attached a script that will create the tables, insert some data
and run the queries to make it easy to try it.

Is there a way to simplify this query and make it more efficient?

Thanks!

Guy.

--
Family management on rails: http://www.famundo.com - coming soon!
My development related blog: http://devblog.famundo.com

Attachment: tags_and.sql
7) Just Someone Restoring a PITR backup
| +1 vote
I have a process for PITR backups running nicely. I'm pretty amazed by the smoothness of it all!...
PostgreSQL - General
[ Profile | Reply to group ] [ Flat  Thread  Threaded ]
I have a process for PITR backups running nicely. I'm pretty amazed by
the smoothness of it all!

Now I'm looking at the retrieval part, and I have something I'm
looking for clarification on.

The documentation say that a recovery.conf file is needed for the
restore. My tests indicate that I can just restore the backup, copy
over the latest WAL files and launch postgres. Is that ok? Can I just
use this way?

I could create the recovery file and have it copy the files, but as
the second machine I am restoring into can always have the DB
reloaded, I wonder if it isn't easier just to copy the files. I
actually rsync my backup directory and my WAL archive directories,
into the second machine.

Bye,

Guy.

--
Family management on rails: http://www.famundo.com - coming soon!
My development related blog: http://devblog.famundo.com
8) Just Someone Re: How to find the latest (partial) WAL file
| +1 vote
Hi Tom, For now I'm using ls with grep: LAST_WAL=3D$(/bin/ls -t1p $WAL_DIR | /bin/grep -v / |...
PostgreSQL - General
[ Profile | Reply to group ] [ Flat  Thread  Threaded ]
Hi Tom,

> If you sort first by mtime and second by file name you should find the
> right one in all cases, ie, take the latest mtime among the
> properly-named files, breaking ties by taking the higher filename.
>
> It'd probably be better if we had a function to report this, but
> you can get along without one.

For now I'm using ls with grep:
LAST_WAL=3D$(/bin/ls -t1p $WAL_DIR | /bin/grep -v / | /bin/grep -v
backup | /usr/bin/head -1)

But a pg function for that would be great.

Regards,

Guy Naor.

--
Family management on rails: http://www.famundo.com - coming soon!
My development related blog: http://devblog.famundo.com
9) Just Someone How to find the latest (partial) WAL file
| +1 vote
What is the best way to find the latest partial WAL file? Based on my tests, using the mtime isn't...
PostgreSQL - General
[ Profile | Reply to group ] [ Flat  Thread  Threaded ]
What is the best way to find the latest partial WAL file?

Based on my tests, using the mtime isn't 100% accurate, as if a
pg_start_backup/pg_stop_backup() operation is run, the .backup file
created might be newer than the last WAL file. It also seems that the
WAL file related to the backup is being updated according to the
mtime. Using a file name is also not 100% accurate, as it seems that
postgres will recycle the archived files, giving them "future" names
before they are used, leaving the directory with files that are older
by name, but not yet with relevant data.

Is there a way to discover what is the real current WAL file? I've
read that one of the todo's for 8.2 is to handle partial files
archiving, but I'm looking for something I can use now.

Regards,

Guy.

--
Family management on rails: http://www.famundo.com - coming soon!
My development related blog: http://devblog.famundo.com
10) Just Someone WAL archiving and deletion of the WAL segments
| +1 vote
I implemented wal archiving and it seems to be working. The segments are being copied by the shell...
PostgreSQL - General
[ Profile | Reply to group ] [ Flat  Thread  Threaded ]
I implemented wal archiving and it seems to be working. The segments
are being copied by the shell script, and in the pg_log file I see
this line:

LOG:  archived transaction log file "000000010000001D00000096"

But the file is still int he pg_xlog directory. In the documentation I
read that it might either delete or recycle the segment files. As I
still see it in the directory, I want to make absolutely sure my
archiving is working correctly.

So my question is: Is what I'm seeing meaning the WAL archiving is
working? Or should I expect the file to be deleted?

Bye,

Guy.


--
Family management on rails: http://www.famundo.com - coming soon!
My development related blog: http://devblog.famundo.com
11) Just Someone A place to post pgbench results
| +1 vote
After exchanging a few emails regarding pgbench in the list, I was thinking it'll be cool to have a...
PostgreSQL - General
[ Profile | Reply to group ] [ Flat  Thread  Threaded ]
After exchanging a few emails regarding pgbench in the list, I was
thinking it'll be cool to have a place to post pgbench resulats from
all over the place. Just so people can get an idea of what others are
getting.

If more people think it's a good idea, I'll be glad to host it as part
of my blog. I'll create a section for that and will update it
everytime results are submitted. If that picks up steam I'll create a
small database and app to make it more automatic.

Let me know if there's interest and I'll take care of it.

Bye,

Guy.

--
Family management on rails: http://www.famundo.com - coming soon!
My development related blog: http://devblog.famundo.com
12) Just Someone Re: Practical limit on number of tables ina single database
| +1 vote
Hi Jim, Actually, most table are VERY small. On each schema I would expect 4 - 5 tables to go over...
PostgreSQL - General
[ Profile | Reply to group ] [ Flat  Thread  Threaded ]
Hi Jim,

Actually, most table are VERY small. On each schema I would expect 4 -
5 tables to go over 1000 records. The rest will be much smaller, with
most at under 100. And aside from 2-3 tables, the activity will be
pretty low (few records a day at the most extreme).

Can I use this to optimize the fsm part in a different way?

Bye,

Guy.


On 3/24/06, Jim C. Nasby <jnasby@pervasive.com> wrote:
> On Fri, Mar 24, 2006 at 11:15:56AM -0800, Just Someone wrote:
> > Hi Jim,
> >
> > On 3/24/06, Jim Nasby <jnasby@pervasive.com> wrote:
> > > You want max_fsm_relations to be greater than select count(*) from pg=
_class where
> > > relkind in ('i','t') *across all databases*. And you want max_fsm_pag=
es to be bigger than
> > > that. That's the only way you can be assured that you'll be tracking =
free space info for
> > > every table.
> >
> > So I need something like 2,000,000 in max_fsm_relations. So
> > max_fsm_pages now need to be 16 times that? That's how I read the
> > postgresql.conf comment.
>
> Hrm... I forgot that FSM allocation is done in terms of CHUNKPAGES,
> which is #defined at 16. So yes, you'd need 32M pages to track freespace
> for all tables. Given that that's 250GB, I guess it won't work terribly
> well... :)
>
> Will any of these tables be very small and not see any real update
> activity? If so, you could possibly do without being able to store FSM
> info for them. Keeping a close eye on the last few lines of vacuumdb -av
> would be key here.
>
> Another possibility is to change CHUNKPAGES in
> include/storage/freespace.h to 1. That means you could get by with 2M
> pages, which is 'only' 16GB.
>
> Perhaps it would be worth considering some alternatives to how the FSM
> works. In particular, it might be worth it to be able to store free
> space info for multiple relations on a single page. Or perhaps allow the
> backend to tablescan very small tables to look for free space.
> --
> Jim C. Nasby, Sr. Engineering Consultant [email protected: j...@pervasive.com]
> Pervasive Software http://pervasive.com work: 512-231-6117
> vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
>


--
Family management on rails: http://www.famundo.com - coming soon!
My development related blog: http://devblog.famundo.com
13) Just Someone Re: Practical limit on number of tables ina single database
| +1 vote
Hi Jim, ss where o be bigger than space info for So I need something like 2,000,000 in...
PostgreSQL - General
[ Profile | Reply to group ] [ Flat  Thread  Threaded ]
Hi Jim,

On 3/24/06, Jim Nasby <jnasby@pervasive.com> wrote:
> You want max_fsm_relations to be greater than select count(*) from pg_cla=
ss where
> relkind in ('i','t') *across all databases*. And you want max_fsm_pages t=
o be bigger than
> that. That's the only way you can be assured that you'll be tracking free=
space info for
> every table.

So I need something like 2,000,000 in max_fsm_relations. So
max_fsm_pages now need to be 16 times that? That's how I read the
postgresql.conf comment.

> Taking a look at the last few lines of a vacuumdb -av would also be insig=
htful, but I'm
> pretty certain that will only show what you need right *now*, not what yo=
u might need in
> the future.

Thanks!

Bye,

Guy


--
Family management on rails: http://www.famundo.com - coming soon!
My development related blog: http://devblog.famundo.com
14) Just Someone Re: Practical limit on number of tables ina single database
| +1 vote