Tuesday, August 19, 2008

Google App Engine application is temporarily over its serving quota


After developing a google app engine application for two months and investing a lot of time in it, and now serving more than 500,000 page views per month I got the following error appear on my website:

"Over Quota

This Google App Engine application is temporarily over its serving quota. Please try again later."


It seems that i had a small mistake which caused runtime exceptions in some of the requests, This small mistake cause my web site to shutdown for several hours!!!!

I really don't understand Google making this error apeear, I understand that they want to secure their servers but why to shutdown an application for several hours, once the error was fixed the server should be activate again.

Now that the server is done, and many users see the error they won't use the application again!

For this reason I really don't recomment you use Google App Engine.

Sunday, June 29, 2008

Google App Engine

One month after I started writing my first Google App Application I had reached more then 1000 Page View per one day!!!



One of the great advanatges of GAE over creating my own Web Server is that robutness of Google servers, My application is up 100% of the time, And after writing the application in correct way without consuming too much CPU per request and handling exceptions the application can be used anytime by the users.



The question is what would happen when GAE would support also Java, I realy don't know how I would be able to convert my application from Python to Java

Friday, June 13, 2008

Google App Engine Tips

  1. Often your application would work just fine locally but after deploying it the application won't work, a reason for that is that when running locally there is no CPU limit but on Google servers there is a CPU limit, My advice is to deploy your application at night this would enable you to test your application live without interfering the users
  2. Check your Logs frequently to see if your application throws exceptions to the user, You should strive to a point that no error logs would be displayed at all.
  3. To delete old entities you can access from "Application Overview" then choose "Data Viewer" and then select the entity click on the top check-box and then click the delete button, once there are no rows the entity would be removed.
  4. Don't upload an application if you havn't fully tested it before locally.

More tips would be posted in the next posts

Tuesday, June 3, 2008

My First Google App Engine Application

Ok, I finally succeed writing the Beta version of my Google App Engine application, This application translate from Hebrew Language to 15 other Langauges - Engish, Russian, Japanesse, Chinesse, Korean. The application use the datastore to hold the dictionary and also it auto-correct words which are misspelled, You can check the application at this url -

http://tirgumchik.appspot.com

Deploying such application myself which would be both robutness and scalable can cost a lost of money, If buying servers and paying for high bandwith internet connection and maintaining the DB and using Load Balancer and so on, I think that it's worth it.

Wednesday, May 14, 2008

Google App Engine in Java Language

As A Java Developer it is very hard to use Google App Engine.
As for now Google App Engine supports only Python Programming language, Trying learning and writing Python languages can be very frustrating when you are used to the good pleasing Eclipse IDE.

Then I found this nice project:

http://code.google.com/p/java2python/

You can write you Google App Engine Application in Java Language and then just convert it to Python and upload it to Google Servers!!!

Enjoy

Sunday, May 11, 2008

Google App Engine Limits

Google App Engine has many unknown limits and resterictions, here are few of them:

  • Data store is limit to 500mb
  • 5,000,000 monthly page views
  • 3 applications per user
  • Application response time to user must be short something like 3 seconds
  • Bulk upload failed to load long strings, I didn't succed upload strings which are more then 40 letters long
  • Unicode charcters can't store in datastore
  • Bulk upload cookie expiration is 15 minutes long, If your upload takes more time then you would need to run bulk upload several times again and again
  • Number of users which can user Goolge App Engine is limit for now
  • It is not possible to rewrite datastore date only to add more rows using bulk upload
  • Google App Engine is limit for now only to Python programming language

Nevertheless Google App Engine works and it has many advantages which are not exists elsewhere, It is ideal for developers which don't have the resource to maintain a complex website for their applications.

Wednesday, April 30, 2008

Google App Engine Work !!!

After hard work and many workarounds I managed to run my first Google App Engine application !!!

And it works great, The performances are amazing, I have more than 10,000 rows in my tables and selecting the rows takes few milliseconds.

I manage to find workaround for the unicode problem by not writing unicodes in the Goole App Engine datastore, I also managed to find workaround for the bulk upload problem by loading the entities one by one and not 10 entities at a time.

So regarding my last post which suggest not to use Google App Engine, I change my mind, You can use it but you would need to work very hard to make it work.

Monday, April 28, 2008

Google App Engine and GWT

First there was GWT - Google Web Toolkit a very nice framework to develop Ajax applications in Java language which then translate to JavaScript, It worked with Tomcat Web Server. Then Google decide to release Google App Engine a service which provide the ability to develop applications in Python language and deploy them on their servers for free and provide datastore which is like DB to store data.

It seems like their is no syncrhonization at all with the two groups GWT and Google App Engine, once Google goes for Java and then for Python, Is there any direction to which Google wants to go to?

In their website they claim that they choosed Pyhton because it is secure language, There is no more secure lagnuage than Java, How would GWT developers would deploy their applications on Google App Engine, would they need now to convert their code to Python? I really confused

Friday, April 25, 2008

Why not using Google App Engine

After worked several days with Google App Engine, I was realy dissapointed, It seems that since it has just released and it is in "Preview Release" it contains some bugs and since Google choose Python as the programming language it make it hard to develop for programmers which don't know well Python, Here are the problems that I come with

  • Python is VERY hard language to develop applications which required Unicode, Maybe for english applications its not so bad language but when you try writing something which is not english you need to find yourself using decode, encode, unicode and other awful methos.
  • It seems that urlfetch method which retrieve data from other urls doesn't support unicode
  • I tried to upload my datastore to the application server, when I do it locally it works, however when I try to upload the datastore remotly on Google Servers it fails.
  • Datastore also doesn't support unicode characters, everything has to be in english
  • Sometimes after uploading the application to the Google Servers it says error 500, However the application works just find locally
  • Python language is the only programming language I know which requires not using Tabs inside the source code and it doesn't use { and } characters to define block but rather it looks on where the line begins. There are much more advance programming languages like Java which could fit more to be used with Google App Engine

In summary I think that I would wait some time before Google App Engine would have more stable versions.

Monday, April 21, 2008

Working with Google App Engine

I just got today invitation to try Google App Engine,
It seems that I'm one of the lucky 10,000 people that got invitation to try it.

Well first I had to admit that I don't know shit about Phyton, Google App Engine is based about Phyton language, However it seems that it is not hard to learn this langauge once you know Php, Java or other programming language.

The big advantage of Google App Engine, is that you don't have to take care about all installing and deploying applications, taking care for performance or load balancing, you get all this things for Free!!!

It seems that it is pretty easy to develop applications using Google App Engine, you just need to install phyton, and Google App Engine SDK, and thats all.

Then you are your phyton application file, and you can test it localy by running GAE (Goolge App Engine) locally.

Once you finished writing you application you just upload it to the Google Server.

The biggest advanatge of GAE in my opinion it is ability to store data and retrive data from the database using GQL Google Query Language which is just like SQL but a bit different, This allow the application writing highly scalabe SQL selects.

Once I would finish writing my application I would publish it here.
Please comment if you have any questions.

Tuesday, April 8, 2008

Google App Engine

Google announced yesterday about their ambitious project "Google App Engine",

The idea for Google is why we should spend so much effort developing applications? Lets make developers outside Google develop applicatons for us. This is the same idea like "YouTube" the users upload the videos for Google.

Each user gets free 500MB of persistent storage and bandwidth for about 5 million page views a month.

Don't hurry to register since the project is limited to the first 10,000 developers, And there are more than 10,000 developers which already registered.

When I tried to check the App Gallery all the applications returned the following error after trying to login: "The requested page is expiered".


Google App Engine

Thursday, March 20, 2008

Build Mashup Application

In this post I would explain how easily is to build mashup web applications, and I would list providers which provide services to build Mashup applications.

Mashup is a Web Application that combines data from several external sources into the application and thereby the development of such application is wasy and fast.

Example of such external sources are


There a lot of other external sources, like RSS feeds and others.

There are also tools which allow to build Mashup applications On-Line, This tools allows to create workflow which get sources from external resources such as in RSS format and merge this dats into one web site, The major software vendors provide such tools -

The disadvantage of mashup applications are

  • They depend on many sources, so if one of this resources is unavailable then all the mashup application becomes unavailable.
  • The performance of such applications is not good since they depend on several outer resources from different locations.
  • If one of the resources change it's API or start to charge for the service your Mashup application would stop working.

Sunday, March 16, 2008

Cache Manager

Actually I took the following Cartoon idea from a real project which I was involved in it. To improve the performance of our application the Architect wanted us to build local cache (We couldn't use Open Source in our project).





















They gave us several months to develop it... :)

Monday, March 10, 2008

Terracotta - Simplfy Cluster Development

Cluster development is often very complicated, Many applicatoins needs to develop special infrastructure to support clustering such as Cluster cache, replication of data between the different servers in the cluster and each develop need to take into account the design and the implementation of his module to run well in cluster environement.

Terracotta comes to solve this problem, Terracota is Open Source and it is called "Network Attached Memory" or simply NAM. Terracotta take care for all the cluster issues, eliminate the developer the need to deal with all this complicated stuff.

For example java synchronized(), wait() & notify() can work exactly the same across several VMs as across threads!!!

Terracotta also provides In Memory DB which is durable to Server Crashes.

In summary if you are looking for an open source which can scale your application in little effort I recommend using Terracotta.

Sunday, March 9, 2008

Execute Async Method

Sometimes you need to execute methods in async way.

There are several ways which you can do it

  1. Send JMS message in J2EE Application with a Callback object, Write MDB which would listen to the Queue which you send the message to, When the MDB would finish it would call the method 'done' on the Callback if the execution succeed, and it the execution failed it would call the method 'failed' on the Callback.
  2. If you write with Java 1.5 simple small Java Program and you don't need to distribute your async command in cluster environment you can use the 'Executors' class to create pool of executions in this way:

ScheduledExecutorService executorService = Executors.newScheduledThreadPool(10);

and then just write:

executorService.submit(callable);

This would execute the callbale call method in async way, You can pass callback to the

callable object which would later be called to send the response back.

Saturday, March 8, 2008

Captcha Hack - Part 1

Captcha is a type of challenge-response test used in computing to determine whether the user is human.





For example:









Does Captcha word can be read by a non-human eyes?

The answer is yes, In this Post I would explain the idea of how this can be done, the algorithm behind it and at last a sample code which implement the algorithm.

Trying to read the letters using image processing is pretty difficult, we would use different technique.

Not all Captcha words can be hack, However the hacking of Captcha code take advantage of the disadvanatges of poor Captcha words.

Many Captcha words are combine of four letters or digits,
Lets assume that it combine both from letters and digits, since abc containts 26 letters and there are 10 digits there are 36 combinations for each letter/digit.

This gives us a total of 36*36*36*36 = 36^4 = 1,679,616 combinations.

If the Captch word is combine only from letter it is even better, this would give us a total of only 26*26*26*26 = 26^4 = 456,976 combinations.

We can then build a DB with all the possible comibnations and give a unique key for each of one of the combinations.

When a Captcha image is displayed we then generate a unique key from this image, and find the matching one in the DB, this would give us an understandable word which can be used to enter the Captcha word.

Some assumptions were done to implement this algorithm:

  • The Captcha word generate must be at the same format, meaning captcha background must have the same pattern, Captch words must be the same font each time.
  • The number of letters/digit can't be over 4 digits, this algorithm wouldn't be able to solve the Captcha word since 36^5=60,466,176 which is too much to store at a simple DB.

The algorithm would be able to solve this special scenarios:

  • If the number of digits is changing between 3 or 4 digits
  • If the backgorund color or the font colors are changing
  • If there are simple lines on the Captch word which intend to interrupt reading the word

In the next posts I would give a more detail algorithm with a solution for the above speical scenarios.

Webmasters - Improve your Captcha words!!!

Monday, March 3, 2008

Debug Web Applications

Debuggubg Web Applications is not an easy task,
However this area is involving, and there several ways now to debug Web Applications.

  1. The simplest way is to use remove tags inside your html/jsp/jsf files and check the result, or adding border="1" to tags and see the layout of your html.
  2. in FireFox you can use tool which is called FireBug, This is a great plugin in FireFix which allows you to debug JavaScript code, It can also show you the border of each component in the HTML, you can also see the HTTP Get and Post requrest between the Client and the Server.
  3. If you develop your Web Application using Google GWT you can debug your Web Application in Develop Mode as it is a pure java application.

Wednesday, February 20, 2008

Typical Software Development Project Cartoon


Thursday, February 7, 2008

80% of commercial apps to use open source by 2012

I had found this intresting article which claim that 80% of commercial apps to use open source by 2012, This are not good news to application software giants such as Oracle, Microsoft, Sun, IBM etc', While for every aspect of software developing you have Open Source alternative the Open Source alternative becomes more appealing for it's robutness, and for it being free.

In IDE enviroment when coming to choose between Eclipse, IntelliJ, Netbeans most choose Eclipse, In Application Server when coming to choose between WebLogic, WebSphere, JBoss and some companies use JBoss.
Tomcat, Apache, GWT all of then are open source.

So how open source projects make their profit? The answer is from support, and not only Open Source projects, many of the software applications giants make their profit from support which can reach up to 80% net incomes.

Only in companies in which the security is critical such as Banks or Goverment institutes Open Source is not being use beacuse it might containg backdoors.

Design Patterns Quick Reference in two pages

I had found this amazing Design Patterns quick reference which containg All the basic design patterns in only two pages!!!

I had printed it in my office and it is very helpful when designing new components,
This great quick reference can be found at mcdonaldland blog.

Enjoy

Links

 
RSS Feeds Submission Directory