Struggling Developers

Grails Tips Part I – Domain Classes

2009-01-28T12:18:00.000-08:00

I have been working a lot with Grails lately and I have to say that I love it!

It has allowed me to create some great web applications in a very fast and reliable way. While developing these applications, I learned a lot of useful things that I would like to share to you. Today I’ll give you some tips regarding Domain Classes.

1) Date created & Last Updated

If you want Grails to handle the date in which an object is created and the last time it was updated, all you have to do is add the following attributes to your domain class:

Date dateCreated
Date lastUpdated

As easy as that! Grails will automatically set the value for this two attributes when an object is created and every time it is updated.

2) Field order

If you want Grails to generate the fields for the create, edit & show views in a particular order, just define that order on the constraints. If the attribute doesn’t require a specific constraint (such as maxSize, unique, nullable, etc), just put the attribute on the constraints and leave the constraint empty.


static constraints = {
    name()
    address()
    city(nullable:true, blank:true)
    phone(nullable:true, blank:true)
    mobile(nullable:true, blank:true)
    email(email:true)
}

3) Unique by group

If you want an attribute on a domain class to be unique, but only grouped by another attribute, you can do it. In the following example, the company name will be unique only for companies from the same client:


class Company {

Client client
String name
String address

static constraints = {
    name(size:3..50, blank:false, unique:'client')
    address()
}
}

4) Change database mapping for String attribute

Grails will map the String attributes on your domain class to varchar(255) by default. If you want a String attribute mapped to text, here’s how you do it:


static mapping = {
    columns{
        attribute_name type:"text"
    }
}

Stay tuned for more Grails tips!

Text encryption on PHP

2008-12-16T13:36:00.001-08:00

A long time ago I had to implement text encryption on Java for some messages that needed to be sent through a socket, and I found a very good encryption algorithm on the net.

Recently I needed text encryption for a similar scenario, but this time the technology was PHP. So, I decided to "translate" this algorithm from Java to PHP and now I want to share it with you, in case you need it.

Encrypt:


function encrypt($text, $seed) {
 $encText = "";
 $pwLength = strlen($seed);
 $i=6;
 for ($x=0; $x < strlen($text); $x++) {
  $i = ($i%$pwLength)+1;
  $a = ord(substr($text, $x, 1));
  $b = ord(substr($seed, $i-1, 1));
  $encText .= ($a ^ $b);
  $encText .= ",";
 }
 return $encText;
}

Decrypt:


function decrypt($text, $seed){
 if(!$text == ""){
  if (strlen($text) == 1)
   return "999999";       

  $pwLength = strlen($seed);
  $strText = "";
  $hold;
  $i = 6;
  $crypt = explode("," ,$text);
  for ($x = 0; $x < sizeof($crypt) - 1; $x++) {
   if (strlen($crypt[$x])>2) {
    $test=0;
    $test = (int) $crypt[$x];
    if ($test == 0) $crypt[$x] = substr($crypt[$x], 1,1);
   }
   $i = ($i % $pwLength) + 1;
   $a = (int)($crypt[$x]);
   $b = ord(substr($seed, $i-1, 1));
   $hold = chr($a ^ $b);
   $strText .= $hold;
  }
  return trim($strText);
 }else{
  return "";
 }
}

Example:


$msg = "This is the best blog!";
$seed = "mySeed";
echo $msg;
$encrypted = encrypt($msg, $seed);
echo $encrypted;
$decrypted = decrypt($encrypted, $seed);
echo $decrypted;

Output:
This is the best blog!
57,17,58,22,69,13,30,89,39,13,0,68,15,28,32,17,69,6,1,22,52,68,
This is the best blog!

I hope this algorithm helps you protect your information!

MySQL full-text search optimization

2008-09-25T15:24:00.000-07:00

A while ago we where developing a classified ads site for one of our most important clients. In this site, every ad belongs to a region (city, country) and a category (e.g. vehicles>buy/sell>cars).

When the visitor accesses the homepage, the region is determined from their IP address so only the ads for that region are displayed (the visitor can change the region) and every search is performed only for the ads on that region as well.

At first everything was working great, but when some regions started to have more than 500.000 ads, that’s when we started to get nervous. Every search on these regions took more than 30 seconds, which is obviously unacceptable. We tried everything: create a full-text field on the ads table, optimized the database queries, but could not bring the search time down to a reasonable amount. At this point I was considering fleeing the country, changing my name and closing all my email accounts so that our client couldn’t find me, and start a new life selling bananas in Brazil. I’m glad I didn’t do it (I think).

After a lot of research over the internet, we came across a miracle. That miracle is called Sphinx Search (http://www.sphinxsearch.com). This is an open-source solution for all your MySQL full-text search problems.

How does it work? You simply install the software on your server, create a configuration file for the table you want to perform the searches on and run the indexing process. This will create a dictionary file that allows you to perform searches in less than a second, on huge tables. You can also provide filters on any of the fields of the table as well as perform sorting on the result set. This software also provides a PHP API so that you can access it from your PHP pages and process the result set.

An important side note: since now all searches are performed on the data dictionary (which is created by a process than can take several minutes, so it’s advisable to run it only once or twice a day) and users can be adding new rows to the original table, the data dictionary will not have all the current data.

How can you solve this issue? Easy, you also create a delta dictionary file with the new rows and schedule a process to rebuild it many times a day (depending on how often new rows are added to your table). This delta dictionary is also included on the same configuration file as the master dictionary, but using a different query.

How do you combine both dictionaries? You can have a process that runs at midnight rebuilding the master dictionary with the overall query (e.g. SELECT * FROM ad) and another that runs every fifteen minutes rebuilding the delta dictionary with the new rows added that day. When the process runs again at midnight to rebuild the master index, the data from the delta dictionary will be incorporated to the master index and the delta index will be reset.

Now every search on the site is working great and we are even using it to obtain all the ads from a specific category without a keyword, because it also takes less than a second. It also allows you to extend the search filtering by fields, grouping and sorting.

Here’s an example configuration file for my classified ads site:


source ads {
    type = mysql
   
    sql_host = localhost
    sql_user = user
    sql_pass = password
    sql_db = myDb
    sql_port = 3306

    sql_query = \
            SELECT \
                    id, title, descr, \
                    catid, UNIX_TIMESTAMP(hw_added) as hw_added, \
                    UNIX_TIMESTAMP(exp_date) as exp_date \
            FROM \
                    ad;
   
    sql_attr_str2ordinal = v_title
    sql_attr_uint = catid
    sql_attr_timestamp = hw_added
    sql_attr_timestamp = exp_date

    sql_query_info = SELECT link_id FROM ec4_ad WHERE link_id=$id
}

source delta : ads
{
           SELECT \
                   id, title, descr, \
                   catid, UNIX_TIMESTAMP(hw_added) as hw_added, \
                   UNIX_TIMESTAMP(exp_date) as exp_date \
           FROM \
                  ad \
           WHERE (TO_DAYS(hw_added) = TO_DAYS(NOW()) ;
}   

index ads {
    source = ads
    path = /home/classifieds/sphinx/main/
    # wordforms = /home/classifieds/wordforms.txt
    # morphology = stem_en
    min_word_len = 3
    min_prefix_len = 0
    min_infix_len = 3
}

index delta : ads {
    source = delta
    path = /home/classifieds/sphinx/delta/
}

indexer {
    mem_limit = 256M
}

searchd {
    port = 3312
    log = /home/classifieds/searchd.log
    query_log = /home/classifieds/query.log
           pid_file = /home/classifieds/searchd.pid
}

I hope this will be useful for you and keeps you from fleeing the country!

Combining PHP and HTML pages

2008-09-10T19:23:00.000-07:00

A while ago, one of my clients gave me the source code for one of his websites for me to make some adjustments. The first thing that shocked me when I was going through this code, was the fact that all of the HTML code was inside the PHP pages.

This is just awful. It’s almost like having a toilet inside of your office (although many of you would like that). PHP pages must only contain the logic of your website and HTML pages must be used for the GUI (Graphical user interface).

There is an easy way to combine PHP and HTML into a single output: Smarty (http://www.smarty.net). Smarty is a template engine which allows you to parse a HTML page from a PHP page. You don’t need to install anything. Just download the PHP code and include it on your project.

Here’s a quick example:

index.php

 
<?php

// Smarty initialization
require_once('Smarty.class.php');
$smarty = new Smarty();

$smarty->display(‘index.html’);

?>

index.html


<html>
<head>
<title>Smarty test</title>
</head>
<body>
This is a test!
</body>
</html>

Of course, there are many other many advanced uses for this tool which you can read about from the Smarty Manual. Here are some examples:

Example 1: Variables

Let’s say you want the title of your page to be a variable which you can change from a PHP script. This is the way to do it using Smarty:

index.php


<?php

// Smarty initialization
require_once('Smarty.class.php');
$smarty = new Smarty();

$smarty->assign(“title”, “My title”);
$smarty->display(‘index.html’);

?>

index.html


<html>
<head>
<title>{title}</title>
</head>
<body>
This is a test!
</body>
</html>

You can also assign arrays or objects and use them on you HTML page as well.

Example 2: Loop sentences

Imagine you want to display a list of all your friends, which are stored in an array. This is how you could do that with Smarty:

index.php


<?php

// Smarty initialization
require_once('Smarty.class.php');
$smarty = new Smarty();

$myFriends = array(“Mike”, “Paul”, “Peter”, “John”);
$smarty->assign(“myFriends”, $myFriends);
$smarty->display(‘index.html’);

?>

index.html


<html>
<head>
<title>My Friends</title>
</head>
<body>
 My friends are:
<ul>
{foreach from=$myFriends item=friend}
<li>{$friend}</li>
{/foreach}
</ul>
</body>
</html>

I hope this brief tutorial was useful for you. If you have any questions, please leave a comment and I’ll try to reply as soon as possible.

SEO Friendly URLs

2008-08-25T18:02:00.000-07:00

Let’s say you have a site which has movie reviews. To display a unique review, you would have a PHP page, for example review.php, which obtains the id of the review and shows the title and description of that review. So, if you want to see the review with id 45, you need to access the following URL:

http://yoursite.com/review.php?id=45

Do you see a problem with this? No? Are you sure? Ok, then I’ll have to explain it. With that type of URL, you are not able to know anything about the review. It could be a review for “Casablanca” just as easily as it could a review for “Ace Ventura: Pet Detective”. And that’s not the biggest problem; it is also a bad SEO URL!

How can you fix this? Use SEO Friendly URLs.

Requirements: Apache (with mod_rewrite module) installed on your server (most hosting services already have this).

Step 1: Insert the following code on you PHP script


function StrToSearchFriendlyURL($s) {
   if(!$s) return 'page';

   $s = strtolower(htmlentities($s, ENT_QUOTES, $GLOBALS['CHARSET']));
   $s = preg_replace('/&(.)(?:acute|cedil|circ|ring|tilde|uml|grave|elig|slash);/', '\\1', $s);
   $s = preg_replace('/\W+/', '_', html_entity_decode($s));
   $s = preg_replace('/_{2,}/','_',$s);
   $s = trim($s,'_');

   if(!$s) $s = 'page';

   return $s;
}

Step 2: Apply the above function to your regular URLs to make them SEO Friendy.

Step 3: When inserting the new URLs on your page, add a dash (-), the review id and html extension at the end.

Example:


<a href="{SEO_friendly_URL}-{id}.html">{$review_title}</a>

Step 4: Create a .htaccess file with the following content.


Options +FollowSymLinks

RewriteEngine On
RewriteRule ^[^-]+-([0-9]+)\.html$ review.php?id=$1 [L]

Step 5: Place the .htaccess file on your site’s document root.

That’s it! The URLs on your site will now look something like this: http://yoursite.com/casablanca-45.html which will be interpreted by Apache like http://yoursite.com/review.php?id=45

More about mod_rewrite: http://httpd.apache.org/docs/1.3/mod/mod_rewrite.html
More about .htaccess files: http://httpd.apache.org/docs/1.3/howto/htaccess.html

Obtaining a visitor’s country from its IP address

2008-08-22T20:10:00.000-07:00

Sometimes, when you are building a site, you may want the content of the homepage or even the language to be used, to change depending on the country from which the visitor is accessing.

How can you do this? It’s actually pretty simple, just follow these steps:

1. Get an IP-Country database. Each row of these databases often has three columns: start and end range for the IP address and the country to which that range is associated (e.g. http://www.ip2location.com). Most of these databases are not free, and the ones that are, are generally a little outdated. I recommend purchasing a database, mainly because you can download a new updated version every month.

2. Import the IP-Country database into the database on your site. The IP-Country databases are generally in the .csv format (comma separted values), so you have to execute a query to import all the values. That query should look something like this:


LOAD DATA LOCAL INFILE '/ip-country.csv'
INTO TABLE ip_country
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
(start_range, end_range, country);

3. Include the following PHP code on the page that you want to use to obtain the Country:

$IPaddr = "";
if (getenv("HTTP_X_FORWARDED_FOR")) {
  $IPaddr = getenv("HTTP_X_FORWARDED_FOR");
} else {
  $IPaddr = getenv("REMOTE_ADDR");
}
$ips = split ("\.", "$IPaddr");
$ip_address = ($ips[3] + $ips[2] * 256 + $ips[1] * 256 * 256 + $ips[0] * 256 * 256 * 256);
$sql = "SELECT country FROM ".TBL_IP_TO_CITY." WHERE ".$ip." BETWEEN IP_FROM AND IP_TO";
$res = $db->query($sql);
$row = mysql_fetch_row($res)
$country = $row[‘country’];

That’s it! You can now obtain the name of the country from which you visitor is accessing from and display the appropriate content.

P.S: If you think Ip2Location.com is paying me for each database bought from this Blog, you are wrong!

P.S2: If you are working at Ip2Location.com, please consider paying me for all the referrals!

About this Blog

2008-08-19T18:20:00.000-07:00

Welcome to the Struggling Developers Blog!

My name is Pablo, I have a small software development company together with two partners, and we have been struggling for about a year to keep our company alive. The three of us used to work as employees on different software companies until we decided to open our own. Since then, we had to deal with different types of projects, technologies and clients.

In this Blog I will tell you about the obstacles that we had to face as a company, with both programming and clients, so that maybe you can learn from our experiences (and the lessons we got from them) and use them on your professional life.

I know I might not have the best solution for every problem, so if you have something to add or a different approach for any of my posts, please leave a comment. I promise I’ll read all of them and (if needed) update my posts with your feedback.