New: LushLush by marceloelke

Updated: RR

MoboKoo

Fameball

Comments are always welcome, here or on soundcloud.
More of my productions here: http://soundcloud.com/marceloelke

Besonders interessant war für mich die Entwicklung des E-Mail-Clients und der Einsatz des Zend Frameworks, auf dem einigeTeile der Website basieren. Mittlerweile bin ich ein Großer Befürworter des ZF. Wir haben sehr viel Zeit durch den Einsatz des Frameworks gespart.

Jeder Benutzer kann sich übrigens bei der Registrierung entscheiden ob er sich mit seiner eigenen E-Mail Adresse anmelden möchte, oder ob er eine kostenlose Adresse von Plusser.de haben möchte. Momentan sind mehrere Web und E-Mail Server für Plusser.de im Einsatz, wobei ich denke dass wir schon bald aufstocken werden müssen. Wir verwenden übrigens mehrere MySQL 5 Server mit Replikation sowie PHP5. Für das Load-Balancing sind zwei BIG-IP’s von F5 zuständig.

So, nun aber back 2 work …

Anyway, today i programmed a little php script that uses a different approach to determine who is the “Subversion King of the Month”. It’s counting the line delta directly from the svn repository using svnlook. So the developer with the most lines added to the repository (not the most commits) is the number one.
Here’s some example output (names are anonymized):

--- Most productive users for 11/2007 ---
1   usera                6319      42.47%
2   userb                5797      38.96%
3   userc                1990      13.37%
4   userd                 773      05.20%
--- Most active commiters for 11/2007 ---
1   usera                  47         47%
2   userb                  34         34%
3   userc                   4          4%
4   userd                   2          2%
-----------------------------------------

The script reads all commits (revisions) for the current month, counts the line delta (how many lines have been added/removed) and the amount of commits.
You can view the highlighted source here or download the script here. Use it on the command line like that:
php -f /path/to/svn/repository

I know that the number of lines comitted may not be the one and only criteria to measure the productivity of a developer. But its an indicator to start with.
So the question is: Are you the “Subversion King” in your company/project?

P.S.
Again, please, dont take this serious ;)

Check out the little demo of the movelogger here.

This technique may be usefull or not. But some use cases could be:

• Instead of eye tracking, use mouse tracking.
• Analyze the usage of ajax enabled websites.
• Spionage and other Bad Things™ (not recommended).

The whole thing is coded in javascript using Prototype and script.aculo.us with some php code on the server side.

The data recorded on the demo website is stored only in the php session on the server and gets deleted automatically, soon after you close your browser. But theoreticaly it would be possible to store that data in some kind of database for further analysis.

Please let me know what you think about this. Do you have any ideas for other uses of this technique ?

SELECT FROM_UNIXTIME(
  FLOOR(
    UNIX_TIMESTAMP('2007-01-01') +
        RAND() *
	(UNIX_TIMESTAMP('2007-01-03')-UNIX_TIMESTAMP('2007-01-01'))
    )
) as random_timestamp;

Or if you prefer a function:

CREATE FUNCTION random_timestamp (start TIMESTAMP, end TIMESTAMP)
RETURNS TIMESTAMP NOT DETERMINISTIC
RETURN FROM_UNIXTIME(
  FLOOR(
    UNIX_TIMESTAMP(start) +
    RAND() *
    (UNIX_TIMESTAMP(end)-UNIX_TIMESTAMP(start))
  )
);

mysql> select random_timestamp('2007-10-01', NOW());
+---------------------------------------+
| random_timestamp('2007-10-01', NOW()) |
+---------------------------------------+
| 2007-10-05 23:07:11                   |
+---------------------------------------+
1 row in set (0.00 sec)

What do you think? Write your thoughts into the comments please. Thanks!

Here are some of my latest tunes. I guess its something between minimal, electro and techno. None of these is finished 100% yet, but you’ll get the idea.

http://pure.rednoize.com/tunes/

By the way, the picture in the background is taken in my favorite club, the water gate in Berlin.
I appreciate any comments !

Thanks
puRe

But if you got some slow count query in your application the best way to increase its performance is to replace / remove it.

So if you are going do to “SELECT count(*) FROM products” the best way, is to have a separated table
that stores the number of products. If you’re inserting a row increment the counter, if you’re deleting a row, decrement it.

Here is some example:

CREATE TABLE counter( number_of_products int(10) DEFAULT '0' NOT NULL);

Increment when you’re adding a new product to the products table:

UPDATE counter SET number_of_products = number_of_products +1;

Decrement when you’re removing a product:

UPDATE counter SET number_of_products = number_of_products -1;

In one of my applications i have the case that i have many reads on a table, but just a few updates / writes to the table. The count was in my case not related to a table, but specific to a row of it.

Example:

CREATE TABLE user (
user_id int(10) DEFAULT '0' NOT NULL,
username int(10) DEFAULT '0' NOT NULL,
...
number_of_purchases int(10) DEFAULT '0' NOT NULL,
);

Scenario:
Just imagine that the number of purchases a user has made is somehow complicated to calculate and query. That’s why we store the number directly in the user table.
We also have a website that produces a lot of reads to this table column, but (unfortunately) its quite seldom that a user purchases something. So using this complicated count query in each website impression would be slow.

What comes handy here is using MySQL 5 triggers.
With triggers we can calculate the number_of_purchases each time a purchase is made by a user and update the users number_of_purchases column.

Example:

delimiter //
create trigger trg_update_user_purchases after update on purchases
 for each row begin
call count_and_update_user_purchases(new.user_id);
 end;
//
delimiter ;

delimiter //
create trigger trg_insert_user_purchases after insert on purchases
 for each row begin
call count_and_update_user_purchases(new.user_id);
 end;
//
delimiter ;

delimiter //
create trigger trg_delete_user_purchases after delete on purchases
 for each row begin
call count_and_update_user_purchases(new.user_id);
 end;
//
delimiter ;

So each time a records in the purchase table is inserted, updated or deleted the user table gets automaticly updated.

Your application gains the performance from selecting a static int field instead of executing complex count queries. Additionally your application is not bloated with unnecessary code to increment and decrement the count column, because the trigger does this automatically for you.

The “URL API” has not been changed. You still may use the parameter “p” for plain text and “xml” for xml output. The new optional url parameter “s” defines the search mode (currently supported are md5 and sha1).

I would greatly appreciate your feedback ;)

Here is an example:
The column text in table text1 has a case-sensitive collation (_cs suffix), the column in text2 has a case-insensitive collation (_ci suffix).

CREATE TABLE text1 (
  `text` varchar(50) character set latin1 collate latin1_general_cs NOT NULL default '',
  UNIQUE KEY uniq_text (text)
) ENGINE=MyISAM
;
INSERT INTO text1 (text) VALUES ('a');
Query OK, 1 row affected (0.00 sec)

INSERT INTO text1 (text) VALUES ('A');
Query OK, 1 row affected (0.00 sec)

CREATE TABLE text2 (
  `text` varchar(50) character set latin1 collate latin1_general_ci NOT NULL default '',
  UNIQUE KEY uniq_text (text)
) ENGINE=MyISAM
;
INSERT INTO text2 (text) VALUES ('a');
Query OK, 1 row affected (0.00 sec)

INSERT INTO text2 (text) VALUES ('A');
ERROR 1062 (23000): Duplicate entry 'A' for key 1

Constraints are also affected by collation:
The queries on table text1 give different results (a and A), the two queries on table text2 result in the same row twice (a).

SELECT * FROM text1 WHERE text = 'a';
+------+
| text |
+------+
| a    |
+------+
1 row in set (0.01 sec)

SELECT * FROM text1 WHERE text = 'A';
+------+
| text |
+------+
| A    |
+------+
1 row in set (0.00 sec)

SELECT * FROM text2 WHERE text = 'a';
+------+
| text |
+------+
| a    |
+------+
1 row in set (0.00 sec)

SELECT * FROM text2 WHERE text = 'A';
+------+
| text |
+------+
| a    |
+------+
1 row in set (0.00 sec)

You can display all available collations using

SHOW COLLATION;

or

SHOW COLLATION LIKE 'latin%';

I installed the Chora cvs/svn viewer today to browse our svn repositories. Beside tons of other problems (especially with Horde) I encountered this one:

Chora displays an error message when trying to browse a subversion repository:
Error svn: Can't check path '/root/.subversion': Permission denied

or in german:

Fehler svn: Kann Pfad '/root/.subversion' nicht prüfen: Keine Berechtigung

After 2 hours of googling (usualy takes 2 minutes) I found the following bug report that gave the useful hint:

http://websvn.tigris.org/issues/show_bug.cgi?id=32

Apache now runs as www, and the “USER” environment is set to “bob”.
However, the “HOME” variable is (incorrectly) set to “/root”.

When PHP calls the svn function, it seems that it picks up the HOME
variable. SVN must probably look for it’s preferences inside
~/.subversion, ~ being the HOME variable (/root). Since it runs as
www, it fails and spits out a permission error. I am not sure if the
USER variable changes something in this.

Because upgrading our subversion version (1.3.2) was not possible, here’s the quick-hack solution:
Edit horde/chora/lib/base.php and add
putenv ("HOME=/var/lib/www");
somewhere.
Replace /var/lib/www with the home directory of your apache user (wwwrun on suse 9.3).

Damn .. i hate such waste of time …

The ip 24.141.55.x has added about 400,000 plain text strings to the database.
It started (21 Aug 2006 12:27:50) with “3QOOk28N” and “NtIDSzQ7″, then went over to dictionary words
like “codex”, “hazard”, “strength” to finaly end the mass attack with “Zyzzogeton” (what ever this is supposed to mean) on 02 Sep 2006 06:57:45. For more then ten days the server had a somehow higher load then usual thanks to this guy (although i did not recognized it before yesterday).

In contrast to the “donor of hashes” mentioned above, the ip 195.114.42.x has requested clear text strings for about 53,800 md5 hashes. The attack started on 02 Sep 2006 04:27:24 and lasted until 12:49 the same day. 53,800 queries in 8 hours is a good score.

So, in reaction to this abuse, the web site sleeps some milliseconds with each query from now on.
Normal users should not be affected by this, but these mass query scripts should be slowed down, at least a little bit.

Actually I do not have anything against such mass queries to my server. However the other services running on the machine should not be impaired.

Conclusion: Thanks to the donor, shame on the leecher.

We’ve put a lot new features into the new version, which i will describe detailed in forthcoming posts.
For now you may read the official press release:

On September 1st, one of the leading international hip-hop battle websites, www.hiphop-battles.com, relaunched to introduce a brand new design and a new free arena for graffiti battles.

Frankfurt am Main, Germany (PRWEB) September 1, 2006 -– With the launch of the second version of www.hiphop-battles.com on September 1st, a third battle arena for graffiti graphic battles was introduced to the website, next to the older arenas for rap and beatbox audio battles. The graffiti arena was a response to the high demand in the hip-hop and graffiti scene, which currently lacks legal ways for competing art skills with graffiti writers from all over the world.

Hiphop-battles.com provides hip-hop fans and artists with free access to over 15,000 legal hip-hop mp3s and a way to vote in ongoing battles in a democratic voting system. The ranking system tells the visitor about “who is hot and who is not” and orients new users, to find the most interesting battles. In its second version, www.hiphop-battles.com also received a total makeover, and shines with a brand new design. Switching to another interface language or to another arena got much easier, and can now be controlled from everywhere on the website.

With the rap, beatbox and the new graffiti battle arena, www.hiphop-battles.com now combines three important elements of hip-hop, and strengthens its position as leading hip-hop battle website.

For more information about the relaunch of hiphop-battles.com or the new online graffiti battle feature, please visit www.hiphop-battles.com/press/, or contact Michael Scheel.

About hiphop-battles.com

www.hiphop-battles.com is a unique online competition platform for international hip-hop artists focusing on audio and graphic battles. With over 11,000 members and more then 8000 battles we have become one of the leading hip-hop competition websites. HipHop-Battles.com enables artists to compete their rap, beatbox and graffiti skills with other artists from around the globe. Hip-hop fans get free access to over 15,000 legal mp3s and the constantly growing community is producing new content on a daily basis.

###

• the name for the source table / view
• the primary key columns of the source table / view
• the desired name for the materialized view

Basically this procedure does the following:

• It drops the materialized view if it already exists
• It creates the materialized view with the structure and data of the source table
• It adds a primary key to the newly created materialized view

However the procedure does not create triggers on the table. This could be an improvement for a next version ;)
To call the procedure use:

CALL create_mview('big_table', 'id', 'view_m_big_table');

And here’s the procedure:

DROP PROCEDURE IF EXISTS create_mview;
DELIMITER //
CREATE PROCEDURE create_mview (IN source_table VARCHAR(150), IN primary_key VARCHAR(150), IN target_table  VARCHAR(150))
LANGUAGE SQL
NOT DETERMINISTIC
CONTAINS SQL
SQL SECURITY DEFINER
MODIFIES SQL DATA
BEGIN
-- declare some variables ...
DECLARE m_view_name VARCHAR(150);
DECLARE create_m_view_sql TEXT;
DECLARE drop_m_view_sql TEXT;
DECLARE add_primary_key_sql TEXT;
--
-- create the sql queries to drop the materialized view, create the tmaterialized view
-- and add a primary key to the materialized view.
SET m_view_name:=concat('view_m_', source_table);
SET @drop_m_view_sql:=concat('DROP TABLE IF EXISTS ',target_table);
SET @create_m_view_sql:=concat('CREATE TABLE ', target_table , ' SELECT * FROM ', source_table);
SET @add_primary_key_sql:=concat('ALTER TABLE ', target_table , ' ADD PRIMARY KEY (', primary_key ,')');
--
-- drop the materialized view if it exists
PREPARE stmt1 FROM @drop_m_view_sql;
EXECUTE stmt1;
--
-- create the table
PREPARE stmt2 FROM @create_m_view_sql;
EXECUTE stmt2;
--
-- add the primary key
PREPARE stmt3 FROM @add_primary_key_sql;
EXECUTE stmt3;
--
DEALLOCATE PREPARE stmt1;
DEALLOCATE PREPARE stmt2;
DEALLOCATE PREPARE stmt3;
--
END; //
DELIMITER ;

You can download the procedure also.

This example demonstrates the use of the MD5 database at http://md5.rednoize.com and AJAX to check password strength during signup on a website. After supplying a username and a password, a md5 hash of the password is generated using Paul Johnston’s md5 javascript library.

The hash of the password (not the password itself) is then sent to http://md5.rednoize.com. If the website returns a result for the given password (hence the hash password combination is stored in the md5 database) it can be regarded as “insecure”. Because the md5 hash of the password, and not the password itself is transferred, no sensitive data will be saved at md5.rednoize.com.

For sure not every password that is not stored in the MD5 database can be considered secure. I recommend adding some extra checks (existence of upper and lowercase characters, numbers, special characters and so on) to increase the password strength.

You can see the example in action here: http://md5.rednoize.com/ajax/

Feel free to use the code in this example any play with it ;)
To implement this on your own webserver you would need some kind of proxy script that redirects the AJAX calls from your own server to the md5 database.

Update:

I recommend saving passwords using salted md5 hashes. Salting in short: “When the user sets a password, a short string called the salt is suffixed to the password before encrypting it; the salt is stored along with the encrypted password so that it can be used during verification. Since the salt is different for each user, the attacker can no longer use a single encrypted version of each candidate password. If the salt is long enough, the attacker must repeat the encryption of every guess for each user, and this can only be done after obtaining the encrypted password record for that user.”

Here’s a little example using salted passwords:

To authenticate users on your website (login) your probably using a SQL statement like this one:
SELECT user_id, username FROM users WHERE passsword = MD5('thepassword');

This is insecure. If someone would know the md5 hash of the password, and the password is weak, it could be “reversed” using the MD5 database.

Use salted passwords to avoid this:
SELECT user_id, username FROM users WHERE passsword = MD5(user_id || 'some_secret_string' || 'thepassword');

Its also possible to store the salt along the password in the database.

The Hamachi website describes how it works:
“Hamachi is a UDP-based virtual private networking system. Its peers utilize the help of a 3rd node called mediation server to locate each other and to boot strap the connection between themselves. The connection itself is direct and once it’s established no traffic flows through our servers.”

The installation procedure is very easy:
In short: download, double-click, join or create a network, invite your friends, done.

After install you’ve got a new network adapter with a 5.x.x.x IP attached to it. Now you can create a “Network” (optionally with a password) and start inviting your friends to join your network. After inviting your friends your see their IP addresses in the “contact window”. It’s now possible to ping them, browse their files or whatever. You can only ping the IP addresses that are in your network / contact list.

The great thing about Hamachi is that it works with almost any network configuration. It doesn’t matter if your behind a firewall or nated. Hamachi is also secure. The complete traffic between the peers is SSL encrypted.

Most people use Hamachi to play multiplayer games using the VPN.
I use Hamachi to grant my friends access to the Apache virtual hosts that are running on my development machine.

The only disadvantage of Hamachi is that the mediation-server is not available. That’s why I only use it for “private” things. To connect to my companies network I’m using OpenVPN which is also quite comfortable.

Today i found a workaround for a feature that was realy missing in MySQL.
Let’s say you got a really complicated query that requires a lot of cpu and time to complete, but the resulting data is quite static. That’s the perfect opportunity to use a “Materialized View”. The Bad Thing™ is, mysql does not support what’s usually known as materialized views. But today i found a nice workaround for MySQL 5.
But first let’s clarify what a materialized view is. I found a good explanation here:
“A materialized view is a stored summary containing precomputes results (originating from an SQL select statement).
As the data is precomputed, materialized views allow for (seemingly) faster dataware query answers.”

Okay, so a materialized view saves the result of a query somewhere (e.g. into another table) else.

In my case i had to calculate a rating of users based on votes from other users (Website of http://www.hiphop-battles.com btw) . I’m doing this with some nested views (also a new feature in MySQL5). The whole calculation and query took 0.5 seconds on my athlon xp 3200 machine. Thats way to much to run this query on each page impression.

The trick is a new feature that was introduced in MySQL 5. From the MySQL Manual:
“In MySQL 5.0, you can create one table from another by adding a SELECT statement at the end of the CREATE TABLE statement:

CREATE TABLE new_tbl SELECT * FROM orig_tbl;

So its possible to create a new table with structure based on an already existing table. The data of the table gets copied also.
To create a table of a very processing intensive view we run the following query:

CREATE TABLE view_m_demo SELECT * FROM view_demo;

To make it little bit better we put this code into a procedure (also a new feature in MySQL5):

DELIMITER //
CREATE PROCEDURE updateDemoView ()
BEGIN
    DROP TABLE IF EXISTS view_m_demo;
    CREATE TABLE view_m_demo SELECT * FROM view_demo;
END; //
DELIMITER ;

Now we can call this procedure using a trigger:

delimiter //
CREATE TRIGGER T_updateDemoView AFTER INSERT ON some_involved_table
  FOR EACH ROW BEGIN
  /* calls the procedure to update out materialized view on each insert */
    CALL updateDemoView ();
END //
delimiter ;

The same can be done with an “AFTER UPDATE” trigger. dont use database triggers if the data in the original tables get updated frequently.
Another option in this case is to update the view time triggerd via cron.
To update the materialized view every minute use:

*/1 * * * * echo "CALL updateDemoView ();"  |  mysql -u user --password=pass database

I hope this helps somebody as much as it helps my projects website loading time ;)
before: 0,48 sec
after: 0,003 sec

I even found out that some crazy people use my website in their scripts and programs. Someone (not me) created a search plugin for mozilla. Another guy created a PHP Class that uses my site. Again another guy created an IRC bot that “reverses” given md5 strings on command … using my site.
The website currently produces ~300 MB traffic per month, so thats okay for now.

I really never expected that this little project attracts so much attention ;)

The tune has been used for 5 MC Battles so far.
Brainfreeze vs mc31 (German)
sebbo vs loony-tune (German)
sebbo vs Rap-O (German)
Lennoxxx vs Scheuerman (German)
MrMef vs Die-anna

The quality of the recorded raps is not allways good but its all just for fun.

Donec pharetra augue id dolor. Quisque et massa. Nullam volutpat consectetuer ante. Maecenas ornare. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Duis vitae eros vitae nulla tincidunt iaculis. Nam nulla. Suspendisse metus augue, dictum vitae, varius a, molestie ut, lorem. Donec malesuada dolor quis tortor. Suspendisse in nibh. Suspendisse id arcu eu massa interdum egestas. Sed libero. Pellentesque sed mauris.

Nullam egestas pede in leo. Aliquam ante urna, blandit ac, tincidunt eu, varius ornare, ligula. Vestibulum mattis gravida felis. Vivamus faucibus orci nec quam. Maecenas aliquam pede sed lacus. Donec at ligula. Vestibulum eget nibh vel nibh tincidunt ullamcorper. Etiam quis turpis sit amet arcu malesuada placerat. Vestibulum eget wisi at arcu feugiat egestas. Integer tellus. Pellentesque vitae pede. Vestibulum id eros. Cras non sapien. Pellentesque turpis urna, rhoncus sed, ultrices eget, bibendum egestas, urna. Phasellus porttitor metus nec lacus. Vivamus hendrerit massa quis tortor. Sed sodales condimentum quam. Integer tempus hendrerit ipsum. Nullam scelerisque est a eros. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae;

Proin tincidunt quam at wisi. Nullam ornare, tellus et eleifend tincidunt, urna magna consectetuer libero, in fermentum felis neque non ante. Phasellus tincidunt nunc quis massa. Integer non enim id ante tincidunt vestibulum. Morbi tincidunt. Proin vulputate ornare ante. Nulla facilisis. Pellentesque eros libero, ultrices quis, fringilla in, interdum et, libero. Cras est. Nunc auctor. Aenean tincidunt.

Donec elementum turpis quis dolor. Nullam a felis ac elit accumsan aliquam. Sed vel pede. Praesent sodales lorem sed augue. Proin eu quam. Proin dolor quam, dapibus eu, ullamcorper a, condimentum et, metus. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Donec a eros. Mauris sit amet lorem vitae neque volutpat fringilla. Mauris dui. Sed mauris sapien, rhoncus eget, sodales a, fringilla a, mauris. Nullam placerat velit et orci. Nam vel lorem eu nunc iaculis tempus.