You can only do correlate subqueries in SELECT and WHERE clauses.

So, a subquery in FROM clause does not have access to the outer select.

select 	"Damage" , "IdAttackerWeapon" as w,
	(select sum(dd)
	from
		(select "Damage", "IdAttackerWeapon", abs(("Damage" - t."Damage")) as dd
		from fact_eventplayerdamage order by dd asc limit 5)
	t1)
from fact_eventplayerdamage t
group by "Damage", w

–

Reza

CREATE OR REPLACE FUNCTION agregar(double precision, character varying, integer, character varying, 

character varying, character varying)

  RETURNS boolean AS
$BODY$
declare
 id ALIAS FOR $3;
 establecidas int :=0;
 agregadas int :=0;
 inicio date :=TO_DATE($5,'YYYY/MM/DD');
 termino date :=TO_DATE($6,'YYYY/MM/DD');
 nulo date:= null;
 mensaje boolean := false;
BEGIN
 select into establecidas from proyectos where id=id;
 agregadas :=  count(id) from etapas where id=id;
 IF agregadas < establecidas THEN
insert into etapas (monto, observaciones, id_proy, nombre,

 fch_inicio, fch_termino, prorroga) values ($1,$2,$3,$4,inicio,termino,nulo);

mensaje:=true;
 END IF;
 RETURN mensaje;
END;
$BODY$
  LANGUAGE 'plpgsql'

Aquí mandamos llamar la función y capturamos el valor

CallableStatement proc = conn.prepareCall("{ ?=call agregar( ?,?,?,?,?,? ) }");
proc.setFloat(2,monto);
proc.setString(3,observaciones);
proc.setInt(4,p);
proc.setString(5,nombre);
proc.setString(6,fch_inicio);
proc.setString(7,fch_termino);
proc.registerOutParameter( 1, Types.BOOLEAN );
proc.execute();
if(proc.getBoolean(1))
response.sendRedirect("lista.jsp?clave="+p);
else
out.println("<h3 id='colora'><center>No se pude agregar,</center></h3>");
proc.close();

En algunas maquinas puede generar un error la jsp en el tipo de dato que se registra, esto se soluciona registrando el tipo de dato Types.BOOLEAN por Types.BIT y ya no habrá problema alguno, y para obtener el valor de retorno podemos seguir haciendo uso del metodo getBoolean(1) ya que la documentacion dice esto: Recupera el valor de un parámetro JDBC bit o booleano como un booleano en el lenguaje de programación Java.

Se você, como eu, instalou o postgresql pelo apt-get logo percebeu que não tinha a senha default para o usuário padrão postgres. Bom, até hoje não sei se existe essa senha (se souber me diga..), porém consegui modifica-lá fazendo o seguinte.

Primeiro acesse o arquivo pg_hba.conf em /etc/postgresql/8.3/main/pg_hba.conf
O arquivo estará com umas linha mais ou menos assim:
# Database administrative login by UNIX sockets -
local   all         postgres                          ident (ou ident sameuser)
# TYPE  DATABASE    USER        CIDR-ADDRESS          METHOD

ident significa que o usuário do banco deverá ser o mesmo do sistema operacional. Entretanto não é isso que queremos e como não sabemos a senha do usuário postgres criado faça:

$ sudo passwd postgres

Forneça a senha para usuário root e em seguida nova senha para o usuário postgres. Feito isso hora de mecher no arquivo pg_hba.conf, faça isso:
# Database administrative login by UNIX sockets -
local   all         postgres                          trust
# TYPE  DATABASE    USER        CIDR-ADDRESS          METHOD

Isso fará como que o banco permita que qualquer usuário do sistema operacional ou da rede que possa acessar o banco tenha acesso como qualquer usuário do banco e que possa acessar qualque banco de dados.

Reinicie o Postgresql:

$ sudo /etc/init.d/postgresql-8.3 restart

Feito isso, agora faça:

$ psql -U postgres

deverá acessar o banco postgres como  usuário postgres sem necessidade de senha.

agora defina uma senha para o usuário postgres, fazendo:

ALTER USER postgres  ENCRYPTED PASSWORD ‘password’;

Saia do psql e no prompt volte a acessar o pg_hba.conf e faça:
local   all         postgres                          md5
# TYPE  DATABASE    USER        CIDR-ADDRESS          METHOD

Com o method md5 irá requisitar uma senha para o usuário do “banco de dados” não do sistema operacional.
Reinicie o postgresql:

$ sudo /etc/init.d/postgresql-8.3 restart

E acesse o banco:

$ psql -U postgres

deverá pedir a senha que foi a criada pelo psql através do comando ALTER USER.

Agora deverá funcionando. Senão, é porque esqueci algo, sendo assim, procure no google o que estiver faltando mas em geral é algo como isso.

Sempre que modificar o arquivo pg_hba.conf reinicie o postgresql.

Para maiores detalhes e outras coisas que não estão aqui, acesse:

http://www.postgresql.org/docs/current/static/auth-pg-hba-conf.html

http://www.postgresql.org/docs/current/static/auth-methods.html#AUTH-PASSWORD

Preface

November, 4th. Release Candidate 1 of Database Designer for PostgreSQL 1.2.9 become available. Among three changes comparing to the last beta there is the one which attracts attention — “Execute Script In Single Transaction (Alt + F9)” functionality added. World community shocked.

“What means added? We thought it always was executed in single transaction…” – resounded from all sides.

November, 14th. MicroOLAP Headquarters. Explanatory mission entrusted to the best agent… Me.

pgAdmin shows the last result set

Right now there is no any opportunity to remember who got the idea about script slicing in SQL Executor. The gist was — each returned result set must be displayed.

Have a look how pgAdmin handles multiple result sets. As you can see only the last is available while others are discarded (we may read about this on the Messages tab).

One more notice. Multiple statements in pgAdmin always executed in the single transaction context. This is not a miracle since PQsendQuery function from client library used.

By the way, the fact that PQsendQuery used give us a hope that someday pgAdmin will handle all result sets.

PgMDD shows all result sets in separate tabs

Just for note, I’m not saying pgAdmin is a dinosaur or something. I like it a lot. This is “must have” tool for sure. I’m using it because of other GUI administration utility absence.

As you probably guessed PgMDD creates separate tab for each result set from the very first release.

There is one more important issue why script slicing was implemented. Database Designer is some kind of ideal world. You may use any names, any functions, any data types for model creating.

But real life is cruel. Generated script must work in any conditions even if some statements may fail, e.g. old server version, non-existent role, lack of privileges for some operations, object with the same name already exists etc. That’s why PgMDD’s SQL Executor should give the developer right of choice — abort execution or proceed anyway.

How it’s made

Let me one phrase before I begin: there is no any SQL parser library (or suite) on the market which suits even the basic needs (I mean PostgreSQL dialect of course). I guarantee this!

God is my witness, our team tried every 3rd party library we meet. Without success.

In Russian speaking IT folklore there is an adage “Переписать всё нахрен!”. Loose translation is “Rewrite all from scratch!”

OK, the moment of glory. We made it ourselves using the native PostgreSQL grammar. Yeah, bite me unbelievers!

Let’s omit technical details. I know nobody cares anyway. Our parser is absolutely… no, I mean absolutely compatible with PostgreSQL. But with 8.3.x version. It’s just a matter of time to update it, but we missed the moment.

So we have two reasons to add “Execute in Single Transaction” functionality:

• Ability to ROLLBACK all changes made by script in case of need
• PgMDD parser cannot proceed with some PostgreSQL 8.4.x features

That’s all folks!

Hello,
if you’re looking for a very nice, flexible, open-source project management tool with great functionality, e.g. sourcecode integration, you have to check-out REDMINE (homepage).
Since I’m running CentOS 5.4 I’ll provide you with the very easy installation procedure:

howto install redmine on centos5.4
=====================
# prerequisites

• enable the epel yum repository
• installed postgresql database (I’m using the latest 8.4 provided from pgdg repository)

# install some packages
yum install ruby ruby-devel ruby-libs ruby-irb ruby-rdoc ruby-postgres rubygems
gem install rails
# create user/db
su postgres
createuser -P redmine (no superuser, can create databases)
psql -U redmine postgres
postgres#>create database redmine;
postgres#>\q
# download latest stable redmine from http://rubyforge.org/frs/?group_id=1850
cd /opt
wget http://rubyforge.org/frs/download.php/67144/redmine-0.8.7.tar.gz
tar -xvzf redmine-0.8.7.tar.gz
# edit ruby database configuration
cd redmine-0.8.7
cp config/database.yml.example config/database.yml
vi config/database.yml
# modify the settings to your recently created database
production:
adapter: postgresql
database: redmine
host: localhost
username: redmine
password: <your redmine database user password>
encoding: utf8

# create a session store (for storing session cookies)
rake config/initializers/session_store.rb
# create the database structure for your configured database in the “production” section
RAILS_ENV=production rake db:migrate
# populate the tables with some initial data
RAILS_ENV=production rake redmine:load_default_data
# set some file permissions (assumption: you’ll run redmine as user redmine)
cd /opt/redmine-0.8.7
sudo chown -R redmine:redmine files log tmp public/plugin_assets
sudo chmod -R 755 files log tmp public/plugin_assets
# adjust your mailserver settings
cp config/email.yml.example config/email.yml
vi config/email.yml
...insert your mailserver/credentials and method of sending emails

# start the webserver as the user you “enabled” in the previous step
ruby script/server webrick -e production
the builtin web-server will start.
Now you can point your browser to http://localhost:3000 and
you’ll see the welcome page, click “sign in” and type credentials “admin”/”admin”.
now you are logged in.

As this admin user you can start creating users, projects, issues, ….
Just play around and have fun, it’s a great tool

any comments highly appreciated….

Ensure your PostgreSQL database is correctly installed and running. Do do it just check the connection using PgAdmin III client. If an error arrises, probabily your server isn’t running. nu Start it using the “Start server” entry in the start menu of your operating system or else on Ubuntu this will work (if the server was installed as in this previous post)

sudo /opt/PostgreSQL/8.4/installer/server/startserver.sh

1) Open the Databases tree item and have a look at the available databases. The postgres database is the user database for the default postgres user and is not too interesting to us. The template_postgis database is what we are going to use to create spatial databases. Right-click on the Databases item and select New Database.

Note: If you receive an error indicating that the source database (template_postgis) is being accessed by other users, this is likely because you still have it selected. Right-click on the PostgreSQL 8.4 item and select Disconnect. You can then double-click the same item to reconnect and try again.

2) Fill in the New Database form with the info below and click OK.

Name: your_database_name

Owner: postgres

Encoding: UTF8

Template: template_postgis

3) Select the new database and click on the SQL query button (or go to Tools > Query Tool).

4) Enter the following query into the query text field:

SELECT postgis_full_version();

5) Click the play button in the toolbar (F5 will work also) to execute the query. The query will return a string that confirms that PostGIS is properly enabled in the database.

Information source

This is a simple method I found to install postgres and postgis in ubuntu/windows. Browse to this url http://www.postgresql.org/download/ and in “Binary Packages” you may choose the operating system where you will be using the server. Choosing windows or linux you will download an executable, but the problem in linux you will have to chmod it in order to use it as one. This will do the trick using the terminal:

  chmod 777 filename.bin

and that’s it. The binary file can now be run:

  sudo ./filename.bin

This will show a GUI installer interface where you will be able to define the root password. The final process will be the selection of addons like database connectors, web components to work with, wikis and the spatial extension we are looking for – postgis. First select the server already running in your OS platform:

In the next screen be sure to check the PostGIS extension.

The installer will then download and install the selected extensions. Open PgAdmin III and be sure that a database called template_postgis exists.

That was a peace of cake… the next post will be about creating a georeferenced enabled database.

Ok, so the INSERT statements were too numerous for inputing the LiDAR point dataset (about a billion points… .)  They kept crashing the postgres daemon.  So, I used copy from a CSV file:

COPY base.cuy_lidar_all FROM 'c:/path/cuy_lidar_ground_veg.insert' WITH CSV

Keep your fingers crossed… .

From time to time there are news/messages about bloated tables in postgres and a thereby decreased performance of the database.
The postgres-wiki contains a view (extracted from a script of the bucardo project) to check for bloat in your database here
For a quick reference you can check your table/index sizes regularly and check the no. of tuples to assume where bloat comes in.
e.g. check size of your tables and indexes:

SELECT relname AS table_name,
pg_size_pretty(pg_relation_size(oid)) AS table_size,
pg_size_pretty(pg_total_relation_size(oid)) AS total_size
FROM pg_class
WHERE relkind in ('r','i')
ORDER BY pg_relation_size(oid) DESC;


check no. of tuples

SELECT relname, relkind, reltuples, relpages
FROM pg_class
ORDER BY relpages DESC;


If you’re facing such a situation you have to check your (auto)vacuum settings and if it’s working correctly.
Starting with Postgres 8.3.x the autovacuum daemon works reliable for most cases and should be enabled. On some workloads a manual vacuum is needed anyhow. In combination with setting the (auto)vacuum parameters you have to consider the Free Space Map (FSM) parameter.
Open your postgresql.conf and check it out (every parameter has a very good comment there):
* enable autovacuum and log every run > 1 sec.

autovacuum = on
log_autovacuum_min_duration = 1000


fine tune when autovacuum will check the tables. The number is the fraction of the table size which needs modified data to run an autovacuum.

Apart from the bloated-table/index stuff, for the query planner it is very important to have up-to-date statistics. If you need an analyze more often, shrink the number to e.g. autovacuum_analyze_scale_factor = 0.02 and the tables will be autoanalyzed more often

autovacuum_vacuum_scale_factor = 0.2
autovacuum_analyze_scale_factor = 0.1

prevent autovacuum from running too often

autovacuum_naptime = 1min #time between two runs


And don’t forget to check your postgresql.log file for any hints the database system provides.
Regarding bloated tables I detected something like:

WARNING: relation "public.table_one" contains more than "max_fsm_pages" pages with useful free space
HINT: Consider using VACUUM FULL on this relation or increasing the configuration parameter "max_fsm_pages"


A “normal” vacuum wouldn’t help in this case because it doesn’t release the pages to the os, they will only be marked as free in the FSM (Free Space Map).
If your FSM isn’t big enough your tables will be bloated even with running autovacuum and perhaps manual vacuum, because the pages with no longer needed data cannot be marked as free and therefore never be overwritten.
One solution would be to run a

vacuum full verbose analyze table_name

This “defragment”-command marks unused pages as “can be overwritten” and moves pages from the bottom into this marked pages. => The table size will shrink
! But be aware: vacuum full locks the table exclusive. Consider running this command in maintenance windows or during night.

You should also keep in mind to “defragment” bloated indexes with

REINDEX INDEX indexname # recreate index explicitly
REINDEX TABLE tablename # recreate all indexes for this table


PostgreSQL

O PostgreSQL é um Sistema Gerenciador de Bancos de Dados Objeto Relacional (SGBD ou SGBDOR) gratuito e de código-fonte aberto que é considerado por muitos o mais avançado SGBD de código-fonte aberto existente na atualidade. Neste post, veremos como instalar e configurar o servidor PostgreSQL e também como instalar e configurar as ferramentas de administração de bancos de dados phpPgAdmin e pgAdmin (pdAdmin3) que podem ser utilizados para acessar um servidor rodando na sua máquina local ou em qualquer outra máquina na sua rede (ou na internet). Em nosso exemplo usaremos um sistema Ubuntu Linux (ou, para ser mais específico, um Kubuntu 9.10), mas os mesmos princípios podem ser utilizadas para a configuração de uma instalação do PostgreSQL em outras distribuições Linux e até mesmo em outros sistemas operacionais.

Instalação e Configuração do PostgreSQL

A maneira mais fácil de se instalar o PostgreSQL no Ubuntu, ou qualquer outra distribuição baseada no Debian, é através dos repositórios de pacotes. Para isso, você pode usar o seu gerenciador de pacotes gráfico (tal como o Adept ou o Synaptic) ou usar o apt-get na linha de comando:
usuario@notebook:~$ sudo apt-get install postgresql

Serão baixados da internet os pacotes necessários e suas dependências e, em pouco tempo, você já terá a última versão do PostgreSQL (atualmente, versão 8.4) instalada em sua máquina. A primeira coisa a fazer após a instalação é configurar a senha do usuário “padrão” do sistema, o usuário “postgres” que é criado com a instalação do PostgreSQL. Para isso, basta usar o cliente de linha de comando “psql” que foi instalado junto com servidor:
usuario@notebook:~$ sudo su postgres -c psql postgres
psql (8.4.1)
Digite “help” para ajuda.
postgres=# ALTER USER postgres WITH PASSWORD ‘123′;
ALTER ROLE
postgres=# \q

Observe que entre os apóstrofos, no lugar de “123″, você deve utilizar a senha que você deseja configurar. Para sair do console do psql pressione simultâneamente Ctrl+D, será ecoado “\q” na tela e você retornará ao prompt do sistema.

Se seu objetivo é apenas usar o usuário padrão do sistema para testar localmente aplicativos que se conectem ao PostgreSQL vocẽ pode parar a leitura desta sessão aqui e pular para a sessão de configuração dos clientes phpPgAdmin e pgAdmin.
Mas, se você está configurando um servidor para uma rede local, talvez seja útil fazer também as configurações que seguem.
Uma outra configuração interessante pode ser liberar o acesso ao SGBD para os usuários comuns da máquina, para isso, localize e edite como super-usuário o arquivo “pg_hba.conf” (no Ubuntu, localizado na pasta “/etc/postgresql/8.4/main” , mas isto pode variar conforme o sistema operacional ou versão do SGBD instalada na sua máquina):
usuario@notebook:~$ sudo kate /etc/postgresql/8.4/main/pg_hba.conf
No lugar do editor “kate” você pode utilizar qualquer outro de sua preferência como o “gedit” ou o “nano”.

Com o arquivo aberto, localize as linhas:
# Database administrative login by UNIX sockets
local all postgres ident sameuser
e substitua a segunda, ficando assim:
# Database administrative login by UNIX sockets
local all all trust
Isso irá liberar a administração local das bases de dados para os usuários comuns da sua máquina.

Se você deseja também liberar o acesso para conexões via rede basta substituir as seguintes linhas (no mesmo arquivo):
# IPv4 local connections:
host all all 127.0.0.1/32 md5
por:
# IPv4 local connections:
host all all 0.0.0.0/0 trust
Note que estas foram configurações bastante genéricas que você talvez precise revisar para aumentar o grau de segurança ou otimização do seu servidor.

Para que estas configurações relativas à rede tenham efeito, você precisa também alterar o arquivo “postgresql.conf”, presente na mesma pasta, para configurar seu servidor para aceitar conexões vindas da rede local. Abra este arquivo, novamente usando o “kate” ou o seu editor de texto preferido:
usuario@notebook:~$ sudo kate /etc/postgresql/8.4/main/postgresql.conf
Localize e substitua a linha:
# listen_addresses = ‘localhost’
por:
listen_addresses = ‘*’
Após alteradas estas configurações é preciso reiniciar o servidor PostgreSQL para que as novas configurações tenham efeito:
usuario@notebook:~$ sudo /etc/init.d/postgresql-8.4 restart
* Restarting PostgreSQL 8.4 database server
Note que a versão do seu servidor instalado pode ser diferente, necessitando que você altere o “8.4″ no final do comando.

Instalação e Configuração do phpPgAdmin

Tela inicial do phpPgAdmin.

O phpPgAdmin é uma ferramenta de administração de banco de dados para a web, por ser escrito em PHP, requer um servidor de páginas web instalado localmente. Ele é bastante popular em servidores de hospedagem, permitindo aos usuários acessar e editar suas bases de dados através do navegador, sem a necessidade de instalar um software localmente. Se você não deseja ter um servidor web completo instalado na sua máquina você pode pular esta etapa e ler sobre a instalação do cliente pgAdmin à seguir. Se você é um desenvolvedor de páginas web é provável que você já tenha o Apache instalado em sua máquina para testes.
Para instalar o phpPgAdmin basta utilizar novamente o apt-get (ou o seu cliente gráfico de instalação de pacotes):
usuario@notebook:~$ sudo apt-get install phppgadmin
Ele irá instalar o phpPgAdmin e as dependências necessárias (incluindo o servidor Apache e o suporte a PHP).
Feito isso, você deve iniciar o servidor Apache:
usuario@notebook:~$ sudo /etc/init.d/apache2 start
* Starting web server apache2
apache2: Could not reliably determine the server’s fully qualified domain name, using 127.0.1.1 for ServerName

Erro no phpPgAdmin: Login desabilitado.

Após, basta abrir o seu navegador e acessar o seu servidor local em http://localhost/phppgadmin
É possível que você receba a mensagem “Login disallowed for security reasons.” (“Login desabilitado por razões de segurança.”) ao tentar realizar login com o usuário “postgres”. Esta mensagem é exibida por uma configuração padrão do phpPgAdmin que desabilita o login de usuários administrativos. Para alterar esta configuração basta editar o arquivo “config.inc.php” localizado no diretório de configurações do phpPgAdmin na sua máquina (no Ubuntu, o arquivo está em “/etc/phppgadmin/config.inc.php”):
usuario@notebook:~$ sudo kate /etc/phppgadmin/config.inc.php
Com o arquivo aberto, localize a linha:
$conf['extra_login_security'] = true;
e altere para:
$conf['extra_login_security'] = false;
Neste arquivo também estão disponíveis outras opções de configuração, como alterar a porta padrão na qual o PostgreSQL irá receber conexões (por padrão, a porta configurada é a 5432).
Alterações feitas, basta recarregar (F5) a página web aberta no seu navegador e fazer login com o seu usuário (ou com o usuário “postgres”).

Instalação do pgAdmin

Tela inicial do pgAdmin3.

O pgAdmin é outra opção de cliente para administrar a sua base de dados, diferente do phpPgAdmin ele é um software desktop que não necessita que um servidor web, como o Apache, esteja instalado na sua máquina. Para instalá-lo no Ubuntu usando o apt-get, digitamos:
usuario@notebook:~$ sudo apt-get install pgadmin3
Após a instalação, basta localizar o atalho do pgAdmin no menu do seu sistema ou usar o comando “pgadmin3” no terminal de comandos para abrir a interface gráfica do mesmo. Nenhuma configuração extra é necessária, basta clicar no ícone com a imagem de um “plug” na barra de ferramentas do programa para configurar a sua nova conexão.

PostgreSQL x MySQL

Aqui abrimos parênteses para tratar de uma dúvida comum à muitos usuários iniciantes: as diferenças entre o MySQL e o PostgreSQL. Ambos são SGBDs de código fonte-aberto e disponíveis gratuitamente para diversas plataformas tais como Linux, Solaris, Windows ou Mac. Podemos dizer, superficialmente, que o MySQL é um SGBD mais “rápido” e com menos recursos avançados (se comparado ao PostgreSQL) usado tipicamente em sistemas Web, enquanto o PostgreSQL é um SGBD mais confiável e robusto, que conta com inúmeros recursos avançados inexistentes no MySQL. Esta comparação é bastante simplista e menos verdadeira a cada nova versão lançada de qualquer um dos dois SGBDs, mas ainda é válida. Uma leve conclusão a respeito do foco de cada SGBD pode ser retirada do slogan presente na página web principal de cada projeto: “PostgreSQL: The world’s most advanced open source database” (“PostgreSQL: O banco de dados de código-fonte aberto mais avançado do mundo”) e “MySQL: The world’s most popular open source database” (“MySQL: O banco de dados de código-fonte aberto mais popular do mundo”).

Finalizando

O objetivo principal aqui foi demonstrar de maneira prática como instalar o PostgreSQL e configurá-lo para funcionar com o seu Ubuntu Linux e para isso não entramos em detalhes quanto ao seu funcionamento ou ao seu uso.
Dúvidas, críticas, sugestões e correções são sempre bem-vindas nos comentários, um abraço!

Acabamos de disponibilizar diversos novos serviços em nosso segmento de cloud computing (computação em nuvem) em nosso portal.

Todos eles estão relacionados a bancos de dados.

Já contávamos com o MySQL Cloud e agora oferecemos serviços Postgres Cloud e Oracle DB Cloud (versão 10g).

Com essas opções, cobruimos praticamente todo o espectro de aplicações que precisam de bancos de dados para funções desde controle de um simples site até aplicações empresariais com grande quantidade de transações e necessidade de alta escalabilidade, alta estabilidade e velocidade, além de granualaridade em configurações.

Todos estes serviços rodam em nossos cluster especializados em bancos de dados, especialmente otimizados para tarefas que envolvem grande quantidade de IO de disco (usamos RAID 5 SCSI), grande quantidade de memória em cada nó e enorme poder de processamento.

Somos capazes hoje, de atender a grandes exigências do mercado corporativo, a custos baixos, aproveitando todo o ganho de escala que a computação em nuvem proporciona.

Alta velocidade na entrega de serviços contratados neste cluster , agilidade na escalabilidade e baixa latência de conexão são outros diferenciais que somos capazes de oferecer dentro de nossa infraestrutura.

Nosso produtos de DB na nuvem agora são:

• MySQL 5 Cloud
• PostgreSQL Cloud
• Oracle 10g Cloud

  Mais dados sobre estes serviços e informações para contratação instantânea em:

http://shop.bsrsoft.com.br/category.php?id_category=15

A unique constraint implemented as a trigger checking hashed geometry seems like a good idea, that is until I applied it to a multi-10GB dataset.  Not surprisingly, it starts off fast on inserts, and slows down a lot as time goes on.  So, I thought I’d approach duplicates another way, by deleting them once they exist.  So for my table:

base.cuy_contours_2

I have hashed my geometry, and added an index:

UPDATE base.cuy_contours_2 SET hash = MD5(ST_AsBinary(the_geom));
CREATE INDEX cuy_contours_2_hash_idx ON base.cuy_contours_2(hash);

I will follow by deleting the duplicate geometries by checking the hashed geometry:

DELETE
 FROM base.cuy_contours_2
 WHERE gid NOT IN
 (SELECT        MAX(dup.gid)
 FROM        base.cuy_contours_2 as dup
 GROUP BY    dup.hash
 );

But, after a very long time, I get an error:

ERROR:  could not read block 356366 of relation 1663/17185/12118368: Permission
denied

So, it’s a  down week, and most of my maps are cached, so we’ll take the postgres service down, and run it in single user mode:

postgres --single -D c:\postgre_data

And rerun the command on one line, per single user mode, with no semicolon:

DELETE FROM base.cuy_contours_2 WHERE gid NOT IN (SELECT MAX(dup.gid) FROM base.cuy_contours_2 as dup GROUP BY dup.hash );

and hope for the best… .

Oh and thanks to :

http://www.postgresonline.com/journal/index.php?/archives/22-Deleting-Duplicate-Records-in-a-Table.html

For help with this one.

If you need to show SQL in JBoss debug screen (console) from your JPA / EJB project:

<properties>
<property name= “hibernate.dialect”value=”org.hibernate.dialect.PostgreSQLDialect” />
<property name= “hibernate.show_sql” value=”true” />
</properties>

in persistence.xml should do the trick.

Tulisan ini menguraikan apa dan bagaimana cara membuat function menggunakan PL/pgSQL. Didalamnya terdapat panduan tahap demi tahap dalam membuat dan menggunakan function menggunakan PL/pgSQL. Setelah membaca tulisan ini, anda diharapkan dapat membuat dan menggunakan fungsi menggunakan PL/pgSQL.
Overview
PL/pgSQL adalah procedural language yang loadable (dapat langsung digunakan tanpa restart server) pada sistem database PostgreSQL. PL/pgSQL didesain dengan tujuan supaya kita dapat membuat sebuah procedural language yang:

• dapat dipakai untuk membuat function atau trigger
• dapat menambahkan sruktur kontrol pada bahasa SQL
• dapat melakukan komputasi kompleks
• semua type, function, dan operator yang didefinisikan user (buatan anda sendiri atau orang lain) dapat diturunkan (inherit)
• dapat dipercaya oleh server
• mudah digunakan

Function yang telah anda buat menggunakan PL/pgSQL dapat digunakan sebagaimana function bawaan yang sudah ada. Misalnya, anda dapat membuat fungsi komputasi kondisional yang kompleks, kemudian menggunakannya untuk membuat operator atau menggunakannya untuk membuat index.

Keuntungan menggunakan PL/pgSQL
SQL merupakan bahasa yang digunakan PostgreSQL, dan begitu pula sebagian besar sistem database relasional (RDBMS: Relational Dtabase Management System) juga menggunakanSQL sebagai bahasa query. Bahasa ini begitu portable (dapat dipakai dimana saja, dan tentu ada sedikit perbedaan antar RDBMS) dan mudah dipelajari.
Namun, setiap pernyataan SQL harus dijalankan secara terpisah oleh server database. Artinya, setiap aplikasi client anda harus mengirimkan setiap perintah SQL pada server satu per satu. Seletah mengirim 1 perintah SQL, client menunggu hasil prosesnya, kemudian menerima hasilnya dan memroses hasil tersebut. Client kemudian melakukan komputasi selanjutnya, dan kemudian mengirim pernyataan SQL berikutnya pada server. Begitu seterusnya. Hal ini tentu menimbulkan beban komunikasi antara server dan client, yang tentu akan menimbulkan beban jaringan, utamanya jika client dan server database anda terpisah.
Dengan menggunakan PL/pgSQL anda dapat mengelompokkan sebuah blok komputasi atau serangkaian query langsung didalam server, yang tentu saja anda memanfaatkan keunggulan procedural language dan kemudahan penggunaan SQL, sekaligus menghemat ongkos komunikasi dan beban jaringan.
beban komunikasi dari server ke client dan sebaliknya dihilangkan
client tidak perlu menerima hasil/data pada saat proses berlangsung, cukup menerima hasil akhir dari proses tersebut, dan siap digunakan untuk proses komputasi berikutnya
dapat mencegah pengiriman query dari client ke server berulang-ulang, karena function dijalankan di sisi server
Hal ini tentu saja meningkatkan performa aplikasi anda secara signifikan. Dalam membuat function menggunakan PL/pgSQL, anda dapat menggunakan tipe data, operator, dan function pada SQL.

Tipe data parameter dan hasil yang didukung
Function yang dibuat dari PL/pgSQL dapat menerima parameter dalam bentuk skalar apa saja (integrer, char, float, dsb) maupun array yang didukung server, dan dapat pula mengembalikan nilai/hasil dengan tipe data tersebut. Function ini juga dapat menerima parameter dan mengembalikan nilai dengan tipe data komposit (tipe row). Function ini juga dapat mengembalikan nilai berupa record yang kolomnya sesuai dengan query yang dijalankan. Function yang dibuat dari PL/pgSQL juga dapat menerima parameter dan mengembalikan nilai bertipe polymorphic (anyelement, anyarray, dsb).
Function dari PL/pgSQL juga dapat dideklarasikan untuk mengembalikan nilai berupa sebuah set atau tabel. Dengan demikian fungsi ini dapat memberikan hasil dengan menjalankan perintah RETURN NEXT untuk setiap elemen dalam 1 record hasil query, atau menggunakan RETURN QUERY untuk mendapatkan semua hasil query.
Function ini juga dapat dideklarasikan untuk menghasilkan nilai void jika anda menghendaki function ini tidak mengembalikan apa-apa.
Function ini juga memiliki kemampuan untuk menerima parameter dengan tipe data berupa output, sehingga anda tidak perlu memberi perintah secara eksplisit untuk mengembalikan sebuah nilai dari suatu function. Ini seringkali berguna jika anda ingin mengembalikan beberapa nilai sekaligus.
Function Anda yang Pertama
Pada bagian ini anda akan saya ajak untuk membuat function, tentu saja function anda yang pertama. Pertama anda harus memiliki software-software yang dibutuhkan. Jika anda meggunakan windows, ada cukup perlu master installer PostgreSQL yang didalamnya sudah memasukkan server PostgreSQL dan aplikasi (client) manajemen database PgAdmin. Jika anda menggunakan linux, gunakan server PostgreSQL dan PgAdmin sesuai dengan distribusi/distro yang anda gunakan. Saya menggunakan PostgreSQL 8.3 dan PgAdmin 1.8.
Buatlah sebuah database untuk mempraktekkan contoh-contoh yang akan saya tampilkan. Buat database baru dengan nama contohplsql. Kemudian jalankan script SQL yang dapat di-download disini, pada database tersebut. Jika sukses, database contohplsql akan memiliki 3 tabel (nama, alamat, negara) dan 2 function (uuid_generate_v4, get_uuid). Kedua function ini hanya digunakan untuk meng-generate primary key pada tiap-tiap record menggunakan algoritma UUID. Klik disini untuk informasi lebih lanjut tentang UUID.
Melalui PgAdmi, masuklah pada database contohplsql, kemudian buka window baru untuk menuliskan script SQL.
[gambar 1]
[gambar 2]
Kemudian tuliskan kode berikut:
CREATE FUNCTION somefunc() RETURNS integer AS $$
<< outerblock >>
DECLARE
quantity integer := 30;
BEGIN
RAISE NOTICE ‘Quantity here is %’, quantity; — mencetak 30
quantity := 50;

/* Create a subblock */

DECLARE
quantity integer := 80;
BEGIN
RAISE NOTICE ‘Quantity here is %’, quantity; — mencetak 80
RAISE NOTICE ‘Outer quantity here is %’, outerblock.quantity; — mencetak 50
END;

RAISE NOTICE ‘Quantity here is %’, quantity; — mencetak 50

RETURN quantity;
END outerblock;
$$ LANGUAGE plpgsql;

Jalankan kode tersebut, anda baru saja membuat function anda yang pertama, dengan nama somefunc. Kemudian panggillah function yang baru saja anda buat dengan menjalankan perintah berikut:
SELECT * FROM somefunc()
Maka anda akan mendapatkan hasil berupa return value bertipe integer bernilai 50. Selamat! Anda baru saja berhasil membuat function.
Struktur PL/pgSQL
PL/pgSQL merupakan bahasa yang dibangun dari blok-blok. Syntax lengkap untuk mendefinisikan sebuah function harus membentuk sebuah blok. Sebuah blok didefinisikan sebagai berikut:
[<>]
[DECLARE
declaration]
BEGIN
statement
END [label];
Pada function pertama yang sudah anda buat tedapat blok yang bernama outerblock. Karena dia didefinisikan dibagian atas, blok ini melingkupi seluruh function. Didalam blok outerblock terdapat blok yang tidak diberi nama (jadi block sebenarnya ditandai dari adanya BEGIN dan END). Setiap deklarasi dan setiap statement dalam sebuah blok harus diakhiri dengan titik koma (;). Sebuah blok yang berada dalam blok lain, tanda titik koma harus ada setelah END. Khusus untuk blok yang menandakan badan function (mencakup keseluruhan isi function) tidak perlu titik koma.
Label hanya diperlukan untuk menandai sebuah blok jika anda menggunakan statement EXIT, atau untuk menandai nama variable yang sifatnya lokal di dalam blok tersebut. Label yang diberikan harus cocok antara yang ada di bagian BEGIN dan END. Pada function contohplsql, block outerblock diakhiri dengan END outerblock;
Semua kata kunci (key word) dibedakan huruf besar/kecil. Identifier secara implisit dikonvesi menjadi huruf kecil, kecuali yang telah diberi petik ganda, sama seperti SQL pada umumnya.
Ada 2 cara untuk menuliskan komentar pada PL/pgSQL. Pertama menggunakan awalan double dash (–), kedua menggunakan awalan /* dan akhiran */.
Setiap statement dalam sesi statement pada sebuah blok dapat ditempatkan pada sebuah sub-blok. Pembuatan sub-blok bisa didasarkan pada logika pemrograman maupun untuk tujuan melokalisir variable. Namun, anda masih bisa mengakses varible diluar blok dengan jalan menggunakan nama label blok tersebut. Perhatikan function pertama yang sudah anda buat. Blok bagian luar bernama outerblock, sedangkan didalam blok ini terdapat sub-blok yang tidak diberi nama. Pada contoh ini, sub-blok digunakna untuk melokalisir varible. Didalam sub-blok ini terdapat variable quantity, dimana namanya sama dengan variable pada blok induknya. Namun karena posisinya berbeda blok, kedua variable ini berbeda. Untuk mengakses variable quantity pada sub-blok cukup menggunakan quantity saja, sementara untuk quantity yang berada pada blok induk harus menggunakan outerblock.quantity.
Catatan: Pada function pertama yang sudah anda buat sebenarnya tedapat blok diluar semua blok yang telah didefinisikan. Blok ini dibuat oleh server secara implisit dan mencakup semua bagian dari blok-blok yang telah anda buat. Blok ini memiliki nama yang sama dengan nama function. Jadi, sebenarnya pada blok inilah anda mendeklarasikan parameter function, begitu juga varible spesial lainnya seperti FOUND. Dengan demikian anda dapat mengakses nama parameter dan variable khusus lainnya menggunakan nama function ini.
Penting untuk memahami dan tidak bingung akan penggunaan BEGIN/END sebagai blok dan sebagai transaction. Pada PL/pgSQL pernyataan BEGIN/END hanya digunakan untuk pengelompokan saja, dia tidak bisa start/commit sebuah transaction. Namun, sebuah blok yang didalamnya terdapat klausa EXCEPTION akan membentuk susunan blok tertentu dimana sebuah rangkaian statement didalamnya dapat di-roll back tanpa mempengaruhi perintah-perintah di blok lain.

Declaration
Semua variable yang digunakan dalam blok harus dideklarasikan dalam bagian deklarasi pada blok tersebut. (Kecuali varible yang digunakan dalam iterasi LOOP, yang akan dideklarasikan secara otomatis sebagai integer). PL/pgSQL dapat memiliki tipe variable sebagaimana yang dimiliki SQL, seperti integer, char, maupun varchar.
Pada function somefunc dideklarasikan 1 variable bernama quantity pada outerblock, dan 1 variable bernama quantity pada sub-blok. Berikut adalah contoh deklarasi variable:
user_id integer;
quantity numeric(5);
url varchar;
myrow tablename%ROWTYPE;
myfield tablename.columnname%TYPE;
arow RECORD;
Syntax umum untuk mendeklarasikan variable adalah sebagai berikut:
name [ CONSTANT ] type [ NOT NULL ] [ { DEFAULT | := } expression ];
Jika dalam deklarasi varible terdapat klausa DEFAULT, maka value awal akan dimasukkan ke dalam variable tersebut. Namun, jika tidak ada klausa DEFAULT, nilaia awal yang dimasukkan dalam variable tersebut adalah SQL NULL. Klausa CONSTANT membuat nilai variable ini selalu tetap, sehingga jika terjadi perintah pengisian nilai pada variable ini akan terjadi error. Klausa NOT NULL membuat pengisian SQL NULL pada variable ini error. Oleh karena itu, jika terdapat salah satu saja dari kedua properti ini, maka sebuah variable harus memiliki klausa DEFAULT. Khusus untuk klausa NOT NULL, nilai awal yang dimasukkan tidak boleh NULL.
Klausa DEFAULT dijalankan setiap kali blok tersebut dijalankan, jadi tidak hanya saat function di-compile. Contoh, jika anda mendefinisikan varible bertipe DATE, kemudian diisi dengan nilai default NOW(), maka isi varible tersebut akan berubah setiap blok dijalankan, bukan hanya saat pertama kali function ini di-compile.
Contoh penggunaan klausa DEFAULT dan CONSTANT:
quantity integer DEFAULT 32;
url varchar := ‘http://mysite.com’;
user_id CONSTANT integer := 10;

Alias untuk Parameter Fungsi
Alias digunakan untuk memberikan nama pada parameter fungsi. Secara default nama parameter adalah angka yang urut sesuai definisi paramter fungsi. Perhatikan contoh berikut:
CREATE FUNCTION somefunc(integer, varchar) RETURNS integer AS $$
Pada contoh diatas, terdapat 2 parameter pada function somefunc, keduanya berturut-turut bertipe integer dan varchar. Secara default kedua parameter tersebut berturut-turut bernama $1 dan $2. Penamaan semacam ini, pada kasus tertentu, menyulitkan anda dalam membuat dan membaca kode function. Untuk itu diperlukan penamaan parameter yang mencerminkan penggunaan parameter tersebut. Proses penamaan ini disebut dengan alias.
Terdapat 2 cara membuat alias pada parameter function. Perhatikan 2 contoh berikut:
CREATE FUNCTION instr(varchar, integer) RETURNS integer AS $$
DECLARE
v_string ALIAS FOR $1;
index ALIAS FOR $2;
BEGIN
– some computations using v_string and index here
END;
$$ LANGUAGE plpgsql;

CREATE FUNCTION sales_tax(subtotal real, OUT tax real) AS $$
BEGIN
tax := subtotal * 0.06;
END;
$$ LANGUAGE plpgsql
Pada contoh pertama, alias dilaksanakan pada bagian deklarasi function. Sedangkan pada contoh kedua, alias dilaksanakan saat pendefinisian parameter. Kedua contoh diatas tidak sepenuhnya identik. Pada contoh kedua, kita dapat membuat referensi sales_tax.subtotal, sedangkan pada contoh pertama fitur ini tidak ada. Hal ini disebabkan karena PL/pgSQL akan membuatkan label khusus secara implisit yang melingkup semua isi function dengan nama label sama dengan nama function, sehingga sales_tax.total nampak pada label ini, sedangkan pada contoh kedua alias parameter berada pada label yang berbeda.
Pada kasus parameter dan return function yang bertipe polymorphic, PL/pgSQL akan secara otomatis membuat variable bernama $0 yang akan menjadi tipe data aktual dari return function tersebut. Ini dapat digunakan sebagai variable yang dapat menampung nilai return dari suatu function, meski kadang ini tidak diperlukan. Selanjutnya, variable ini dapat diberi alias seperti parameter, yang telah dijelaskan sebelumnya. Perhatikan contoh berikut:
CREATE FUNCTION add_three_values(v1 anyelement, v2 anyelement, v3 anyelement)
RETURNS anyelement AS $$
DECLARE
result ALIAS FOR $0;
BEGIN
result := v1 + v2 + v3;
RETURN result;
END;
$$ LANGUAGE plpgsql;

CREATE FUNCTION add_three_values(v1 anyelement, v2 anyelement, v3 anyelement,
OUT sum anyelement)
AS $$
BEGIN
sum := v1 + v2 + v3;
END;
$$ LANGUAGE plpgsql;
Pada contoh pertama, alias dilaksanakan pada bagian deklarasi function. Pada contoh kedua, alias dilaksanakan pada pendefinisian parameter.

Menyalin Tipe Data
variable%TYPE

Anda dapat mendefinisikan sebuah variable dengan tipe data yang dinamis, artinya tipe data variable tersebut dapat berubah saat function dijalankan (on the fly). Tipe data seperti ini banyak dibutuhkan jika anda ingin membuat variable yang menampung data dari sebuah kolom pada suatu tabel. %TYPE akan menyajikan tipe data atau tipe kolom pada sebuah tabel. Jika anda ingin membuat variable yang bernama user_id dan digunakan untuk menampung nilai kolom user_id pada tabel user, maka syntax yang digunakan adalah:
user_id user.user_id%TYPE;

Jika kolom user_id bertipe integer maka variable user_id juga beritipe integer. Jika dikemudian hari anda mengubah kolom ini sehingga bertipe float, maka variable user_id juga akan bertipe float, tanpa harus mengganti function ini.

Row Type
name tablename%ROWTYPE;
name composite_type_name;

Sebuah variable komposit adalah variable row (atau variable row-type). Sebuah variable row dapat menampung nilai dari seluruh kolom hasil perintah select. Setiap kolom dapat diakses menggunakan tanda dot (.), misalnya: variablerow.namafield.

Setelah lama tidak mainan PostgreSQL saya jadi terlupa beberapa perintah dasar it. Salah satunya menampilkan tabel.

\dt

Sedangkan untuk nge-dump database bisa pakai perintah di bawah. Pretty simple. Perintah ini akan nge-dump baik struktur maupun data.

$ pg_dump nama_database > out_file.sql

Have been very busy at work lately.  We made the decision about a month ago to switch (most|all) new projects over to use Python 3 with Apache, mod_wsgi, and AppStruct.  You may know what the first 3 are, but the 4th??

Special thanks goes to Graham Dumpleton behind mod_wsgi, and James William Pye behind Python>>Postgresql.   They are not involved or affiliated with AppCove or AppStruct (aside from great mailing list support) BUT if it were not for them, this framework would not exist.

AppStruct is a component of Shank, a meta-framework.  A stand-alone component in it’s own right, it represents the AppCove approach to web-application development.  Most of it is in planning, but the parts that have materialized are really, really cool.

Briefly, I’ll cover the two emerging areas of interest:

AppStruct.WSGI

This is a very pythonic (in my opinion) web application framework targeted toward Python 3.1 (a challenge in itself at this point).  We really wanted to base new development on Python 3.1+, as well as PostgreSQL using the excellent Python3/PostgreSQL library at http://python.projects.postgresql.org/.  However, none of the popular frameworks that I am aware of support Python 3, and most (if not all) of them have a lot of baggage I do not want to bring to the party.

Werkzeug was the most promising, but alas, I could not find Python 3 support for it either.  In fact, I intend to utilize a good bit of code from Werkzeug in finishing off AppStruct.WSGI.  (Don’t you just love good OSS licences?  AppStruct will be released under one of those also).

HTTP is just not that complicated.  It dosen’t need bundled up into n layers of indecipherable framework upon framework layers.   It doesn’t need abstracted to death.  I just needs to be streamlined a bit (with regard to request routing, reading headers, etc…).

Python is an amazing OO language.  It’s object model (or data model) is one of (if not the) most well conceived of any similar language, ever.  I want to use that to our advantage…

Inheritance, including multiple inheritance, has very simple rules in Python.  Want to use that as well.

Wish to provide developers with access to the low level guts they need for 2% of the requests, but not make them do extra work for the 98% of requests.

Speed is of the essence.  Servers are not cheap, and if you can increase your throughput by 5x, then that’s a lot less servers you need to pay for.

So, how does it work?

Well, those details can wait for another post.  But at this point the library is < 1000 lines of code, and does a lot of interesting things.

• A fully compliant WSGI application object
• 1 to 1 request routing to Python packages/modules/classes
• All request classes derived from AppStruct.WSGI.Request

The application maps requests like /some/path/here to objects, like Project.WSGI.some.path.here.  If there is a trailing slash, then the application assumes that the class is named Index.  The object that is found is verified to be a subclass of AppStruct.WSGI.Request, and then…

Wait!  What about security?

Yes, yes, very important.  A couple things to point out.  First, the URLs are passed through a regular expression that ensures that they adhere to only the characters that may be used in valid python identifiers (not starting with _), delimited by “/”.  Second, the import and attribute lookup verify that any object (eg class) found is a subclass of the right thing.  And so on and so forth…

But you may say “wait, what about my/fancy-shmancy/urls/that-i-am-used-to-seeing?  Ever hear of mod_rewrite?  Yep.  Not trying to re-invent the wheel.  Use apache for what it was made for, not just a dumb request handler.

What about these request objects?

They are quite straightforward.  There are several attributes which represent request data, the environment, query string variables, post variables, and more.  There is a .Response attribute which maps to a very lightweight response object.

Speaking of it — it has only 4 attributes: [Status, Header, Iterator, Length].  As you see, it’s pretty low-level-wsgi-stuff.  But the developer would rarely interact with it, other than to call a method like .Response.Redirect(‘http://somewhere-else.com’, 302)

Once the application finds the class responsible for handling the URL, it simply does this (sans exception catching code):

RequestObject = Request(...)

with RequestObject:
   RequestObject.Code()
   RequestObject.Data()

return RequestObject.Response

Wow, that’s simple.  Let me point out one more detail.  The default implementation of Data() does this:

def Data(self):
   self.Response.Iterator = [self.Text()]
   self.Response.Length = len(self.Response.Iterator[0])

So the only thing really required of this Request class is to override Text() and return some text?  Yep, that simple.

But typically, you would at some point mixin a template class that would do something like this:

class GoodLookingLayout:
   def Text(self):
      return ( 
         """<html><head>...</head><body><menu>""" +
         self.Menu() +
         """</menu><div>""" +
         self.Body() +
         """</div></body></html>"""
         )

And then it would be up to the developer to override Menu and Body (each returning the appropriate content for the page).

Ohh, you may say.  What about templating engine X?  Well, it didn’t support Python 3, and I probabally didn’t want it anyway (for 9 reasons)…  If it’s really, really good and fits this structure, drop me a line, please.

What about that Code() method?

Yeah, that’s the place that any “logic” of UI interaction should go.  I’m not advocating mixing logic and content here, but you could do that if you wanted.  You will find in our code, a seperate package for application business logic and data access that the request classes will call upon.  But again, if you are writing a one page wonder, why go to all the trouble?

The only requirement for the Code() method is that it calls super().Code() at the top.  Since the idea is that the class .Foo.Bar.Baz.Index will inherit from the class .Foo.Bar.Index, this gives you a very flexible point of creating .htaccess style initialization/access-control code in one place.  So in /Admin/Index, you could put a bit of code in Code() which ensures the user is logged in.  This code will be run by all sub-pages, therefore ensuring that access control is maintained.  Relative imports are important for this task.

from .. import Index as PARENT
from AppStruct.WSGI.Util import *

class Index(PARENT):
   def Code(self):
      super().Code()
      self.Name = self.Post.FirstName + " " + self.Post.LastName
      # some other init stuff
   def Body(self):
      return "Hello there " + HS(self.Name) + "!"

Summary of AppStruct.WSGI…

To get up and running with a web-app using this library:

1. A couple mod_wsgi lines in httpd.conf
2. A .wsgi file that has 1 import and 1 line of code
3. A request class in a package that matches an expected URI (myproject.wsgi.admin.foo ==  /admin/foo)

Speed?

With no database calls, it’s pushing over 2,000 requests per second on a dev server.  With a couple PostgreSQL calls, it is pushing out ~ 800 per second.

AppStruct.Database.PostgreSQL

This is not nearly as deep as the WSGI side of AppStruct, but still really cool.  To start off, I’d like to say that James William Pye has created an amazing Postgresql connection library in Python 3.  I mean just amazing.  In fact, almost so amazing that I didn’t want to change it (but then again…)

What we did here was subclass the Connection, PreparedStatement, and Row classes.  (well, actually replaced the Row class).

Once these were subclassed, we simply added a couple useful features.  Keep in mind that all of the great functionality of the underlying library is retained.

Connection.CachePrepare(SQL)
Connection._PS_Cache

Simply a dictionary of {SQL: PreparedStatementObject} that resides on the connection object.  When you directly or indirectly invoke CachePrepare, it says “if this SQL is in the cache, return the associated object.  Otherwise, prepare it, cache it, and return it”.  This approach really simplifies the storing of prepared statement objects in a multi-threaded environment, where there is really no good place to store them (and be thread safe, connection-failure safe, etc…)

Connection.Value(SQL, *args, **kwargs)
Connection.Row(SQL, *args, **kwargs)

These simple functions take (SQL, *args, **kwargs) and return either a single value or a single row.  They will raise an exception is != 1 row is found.  They make use of CachePrepare(), so you get the performance benefits of prepared statements without the hassle.  More on *args, **kwargs later under PrePrepare()

Connection.ValueList(SQL, *args, **kwargs)
Connection.RowList(SQL, *args, **kwargs)

Same as above, except returns an iterator (or list, not sure) of  zero or more values or rows.

Row class

Simply a dictionary that also supports attribute style access of items.  After evaluating the Tuple that behaves like a mapping, I decided for our needs, a simple dict would be a better representation of a row.

Connection.PrePreprepare(SQL, args, kwargs) -> (SQL, args)

Ok, so what’s the big deal?  Well, the big deal is that we don’t like positional parameters.  They are confusing to write, read, analyze, and see what the heck is going on when you get about 30 fields in an insert statement.  Feel free to argue, but maybe I’m not as young as I once was.

We do like keyword parameters.  But postgresql prepared statement api uses numeric positional parameters.  Not changing that…

The most simple use case is to pass SQL and keyword arguments.

"SELECT a,b,c FROM table WHERE id = $id AND name = $name"
dict(id=100, name='joe')

It returns

"SELECT a,b,c FROM table WHERE id = $1 AND name = $2"
(100, "joe")

Which is suitable for passing directly into the Connection.prepare() method that came with the underlying library.

I don’t know about you, but we find this to be very, very useful.

If you pass tuples of (field, value) as positional arguments, then they will replace [Field], [Value], and [Field=Value] (in the SQL) with lists of fields, lists of values (eg $1, $2), or lists of field=values (eg name=$1, age=$2).  That really takes the verbosity out of INSERT and UPDATE statements with long field lists.

Conclusion

This is just in the early stages, and has a good deal of polishing to be done (especially on the WSGI side).  My purpose here was to introduce you to what you can expect to get with AppStruct, some of the rationale behind it, and that it’s really not that hard to take the bull by the horns and make software do what you want it to (especially if you use python).

Feel free to comment.

Found a comprehensive research called “Comparison of different SQL implementations”.

Great work done.

Web development today

Web development today isn’t just HTML editing or basic php scripts. Web development and programming today is usually achieved with a wide set of tools — any popular website around today uses a wide set of tools. For example,

• reddit.com — reddit uses Python and PostgreSQL, which spit out html (of xhtml syntax), javascript and CSS.
• slashdot.com — slashdot.com uses Perl and MySQL, sitting on top of Apache.
• microsoft.com — microsoft.com uses a combination of ASP.NET, running on top of IIS. The output is html (xhtml syntax), css and javascript.

The tools

Web Developer / Firefox Extension

Firefox Web Developer Extension is a useful tool for being able to disable various features on a page, manage cookies, view specific css information using a picker tool for any specific elements and being able to manager userstyles. You can also manage images and various other tools including the ability to view generated source.

You can download it at Mozilla Add-ons

ColorZilla / Firefox Extension

ColorZilla is an extension to firefox that adds a color picker tool to the browser UI. It is then possible to return information for a given pixel on a website.

You can download it at the Mozilla Add-ons site.

MeasureIT / Firefox Extension

The measureIT Firefox extension adds a measuring tool that appears as an overlay on any webpage you need to use it on.

You can download measureIT at the Mozilla Add-ons site

DOM Inspector / Firefox Extension

The Document Object Model Inspector is a tool to inspect the DOM of any webpage. It often comes with the default install of firefox as an option.

You can download DOM Inspector at https://developer.mozilla.org/en/dom_inspector

Firebug / Firefox Extension

Firebug is an extension which is useful for editing css, html or javascript elements within a webpage.

You can download Firebug at the Mozilla Add-ons site.

Google page speed / Firebug Extension

Google page speed is a Firebug extension that is able to give you a wealth of information on the download speeds and execution time of your website. It can also give you tips on how to reduce overheads and improve code.

You can download the Google page speed Firebug extension at the google page speed site.

Notepad++

Notepad++ is a tool I often use when working on smaller projects where I do not need a full IDE.

Notepad++ is a free (as in “free speech” and also as in “free beer”) source code editor and Notepad replacement that supports several languages. Running in the MS Windows environment, its use is governed by GPL License.
Based on a powerful editing component Scintilla, Notepad++ is written in C++ and uses pure Win32 API and STL which ensures a higher execution speed and smaller program size.

You can download it over at the notepad++ sourceforge project site.

Netbeans with PHP Development plugin

Netbeans is a Java IDE developed by Sun. It supports various plugins — one being PHP development extensions. When working on large projects, Netbeans proves useful.

You can download it at the Netbeans site.

Visual SVN

VisualSVN is a SVN server for Windows. It is quick and easy to use and requires little previous knowledge of server administration or svn experience, but it can also be quite powerful.

You can download it at the VisualSVN Server site. It is free for both personal and commercial use.

Tortoise SVN

TortoiseSVN is an SVN client that interfaces with Windows Explorer to interface with SVN.

You can download it at the TortoiseSVN site.

XAMPP

XAMPP is a quick and easy way to get an apache server, MySQL daemon and PHP all running on Windows for a development environment. It also supports an FTP server and a few other things.

You can download it at the XAMPP site.

Forums
You can come and discuss web development, get website reviews and also SEO information at the Transcendental Labs forum.

This is how you can get the median value of a column in PostgreSql. Thanks to http://old.nabble.com/SELECT-TOP–d-PERCENT,-or-SELECT-…-LIMIT–d-PERCENT—td19941366.html

WITH c AS (select count(*)/2 as n from FOO) select COL from FOO order by COL asc limit (1) offset (select n from c);

It is much simpler if you are using SQL Server as described in: http://www.sqlmag.com/Article/ArticleID/49827/sql_server_49827.html

SELECT MAX(Value) FROM
  (SELECT TOP 50 PERCENT Value FROM dbo.VOrders ORDER BY Value) AS H1;

--
Reza++