Jaslabs: High performance Software

High Performance Software

Archive for the 'tech' Category

why do open source projects have so many bugs?

By Justin Silverton

In a recent article:

“In his first “meet the press” session as ICT Minister, Professor Sitthichai Pokai-udom put forth his radical vision for 3G in Thailand, condemned open source for turning out buggy, useless software and promised to make the civil servants in the ICT Ministry proud of their organisation once again.

Speaking at the IT Press Club, Minister Sitthichai said that he was looking for an investment of around 20 billion baht for Thai Mobile and between 10 and 18 billion for CAT in order to complete the roll-out of next generation communication networks for the two former state enterprises.

Sitthichai said on detailed questioning that Thai Mobile would upgrade its network and operate a WCDMA 3G network while CAT Telecom would roll out a CDMA2000 EV-DO network.

He said that rather than using public money to compete with the private sector, he expected today’s private mobile operators to gladly become partners to CAT and TOT and effectively lease their networks and focus on the marketing aspect. He reasoned that none of the incumbents would want to invest in a 3G network as 4G would come along in five years and render today’s investments obsolete.”

While I don’t completely agree with the stance that all open source applications are bug-ridden and useless, he does make a good point. I have used many open source applications over the years and many were either bug-ridden, barely functional, or they lacked basic functionali.

Open source sounds like a great concept. The idea is that since the code is open, there is a possibility for more programmers to see the code and create new functionality or fix bugs. This does happen to some degree, but because of the overwhelming number of contributers, many of the projects are subjected to in-fighting, politics, and eventual destruction.

While this is a part of any project, there needs to be more organization for a project to be successful. This involves assigning leaders and only accepting code into the main codebase that meets a certain standard. Many of the larger and more popular projects are using this method and have less bugs as a result (and are around much longer).

If more open source projects followed better project management practices, We would see much better projects an it would benefit both the users and the programming comunity.

Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • Digg
  • del.icio.us
  • DZone
  • Slashdot
  • StumbleUpon
  • Technorati
2 comments

Top 5 freelance websites

By Justin Silverton

The following are the top 5 websites that I have found for freelance work.  This is for mostly programming/development, which includes PHP and many other languages.

5) http://www.rentacoder.com

Rentacoder is good place to find small to mid-sized projects for earning extra money.  It also offers a rating system which can hurt or help you depending on your work.

4) http://odesk.com/

Odesk is a little bit different than the rest of the sites listed here.  Instead of bidding on projects, you bid on hours.  Employers can list the amount of hours/week and # of weeks they need out of a potential freelancer.

3) http://www.craigslist.com

Craigslist is great because there is no barrier to entry for either the employer or the potential freelancer/employee (it is also among the top sites on the Internet).  This can also be a detriment because anyone with a computer can post a job listing.

2) http://programmermeetdesigner.com/

A unique website that provides an opportunity for programmers to meet and work with designers.

1) http://www.guru.com

Guru.com is a clean and professional website that has lots of large-scale projects.  It also has a fairly high barrier to entry for the employers, which usually means better paying jobs.  

Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • Digg
  • del.icio.us
  • DZone
  • Slashdot
  • StumbleUpon
  • Technorati
9 comments

How efficient is your favorite language?

By Justin Silverton

How does you favorite language compare? Below is the results of a series of tests performed on various programming languages. The weights of each specific test are listed in the second chart below.

Language Implementation Score Missing Failing Avg.Score
Delphi delphi 553 7 0 30.72
C vc 665 2 1 30.23
C bcc 665 3 0 30.23
Ocaml ocaml 665 0 3 30.23
C lcc 663 3 0 30.14
C mingw32 663 1 2 30.14
Mercury mercury 475 7 2 29.69
C gcc 674 0 2 29.30
Lua lua5 638 2 1 29.00
Ocaml ocamlb 630 0 3 28.64
Java java 710 0 0 28.40
Forth gforth 595 1 3 28.33
SML smlnj 479 0 8 28.18
Lua lua 648 2 0 28.17
Pike pike 675 0 1 28.13
Ada gnat 447 8 1 27.94
Eiffel se 502 5 2 27.89
Nice nice 556 5 0 27.80
Common Lisp poplisp 389 0 11 27.79
Pascal fpascal 305 8 6 27.73
C++ vc++ 442 5 4 27.63
Pascal vpascal 465 8 0 27.35
C# csharp 574 4 0 27.33
Python python 648 0 1 27.00
Pliant pliant 539 4 1 26.95
S-Lang slang 505 6 0 26.58
ICI ici 610 0 2 26.52
Icon icon 422 9 0 26.38
Modula-2 modula2 209 15 2 26.13
Perl cygperl 595 0 2 25.87
Awk awka 484 6 0 25.47
Awk mawk 483 6 0 25.42
ElastiC elastic 151 16 3 25.17
Oz oz 423 6 2 24.88
Perl perl 560 0 2 24.35
Haskell ghc 364 4 6 24.27
Forth bigforth 239 9 6 23.90
Parrot parrot 238 14 1 23.80
Erlang erlang 319 3 8 22.79
Modula-3 modula3 109 20 0 21.80
Awk gawk 391 6 1 21.72
Tcl tcl 426 3 2 21.30
PHP php 294 9 2 21.00
Simula cim 104 20 0 20.80
Scheme guile 418 0 4 19.90
JavaScript jscript 236 10 2 18.15
Ruby ruby 356 0 4 16.95
Rexx rexx 202 7 4 14.43
VBScript vbscript 145 5 7 11.15
REBOL rebol 0 18 7 0.00
Languages that compile to native code are in Bold Italics.

Weights used for the scorecard
Test Weight Test Weight
Ackermann’s Function 1 Array Access 3
Count Lines/Words/Chars 3 Echo Client/Server 5
Exception Mechanisms 1 Fibonacci Numbers 2
Hash (Associative Array) Access 1 Hashes, Part II 4
Heapsort 4 Hello World 1
List Operations 3 Matrix Multiplication 3
Method Calls 5 Nested Loops 4
Object Instantiation 5 Producer/Consumer Threads 1
Random Number Generator 3 Regular Expression Matching 4
Reverse a File 4 Sieve of Erathostenes 4
Spell Checker 4 Statistical Moments 2
String Concatenation 2 Sum a Column of Integers 3
Word Frequency Count 5    

(Chart reference: http://dada.perl.it/)

Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • Digg
  • del.icio.us
  • DZone
  • Slashdot
  • StumbleUpon
  • Technorati
No comments

Flickr.com - PHP/mysql case study

Introduction

Carl Henderson from Flickr.com, a very popular photo blogging service has released a pdf (not sure exactly when this was actually released) detailing the issues they faced with having a high-traffic website.

original PDF can be downloaded here

some interesting points taken from this pdf are below.

Classes, libraries, and systems used

1) smarty for templating
2) PEAR and XML for Email parsing
3) perl for controlling
4) imagemagick for image processing
5) mysql (4.0/innoDb)
6) java, for node service
7) apache 2 and redhat linux

8) 60,000 lines of PHP code
9) 60,000 lines of templates
10) 70 custom smarty functions/modifiers
11) 25,000 DB transactions/second at peak
12) 1000 pages per second at peak

unicode support

1) UTF-8 pages
2) CJKV support

Tips: don’t use HtmlEntities(). Also, Javascript has patchy Unicode Support

Why php was used

1) Everything can be stored in the database, including smarty cache
2) a “shared nothing” approach (as long as php sessions were not used)

Mysql usage

Select’s: 44,220,588
Insert’s: 1,349,234
update’s: 1,755,503
delete’s: 318,439
13 select’s per Insert, Update, and delete

Tips: many of the tables that needed to be full-text searched were de-normalized. This does waste space, but because it allowed for little or no joins, it made searching much faster.

Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • Digg
  • del.icio.us
  • DZone
  • Slashdot
  • StumbleUpon
  • Technorati
3 comments

10 database speed tests

By Justin Silverton

I came across the following 10 benchmark tests covering:

SQLite version 3.3.3
SQLite version 3.3.3
SQLite version 2.8.17
SQLite version 2.8.17
PostgreSQL version 8.1.2
MySQL version 5.0.18
FirebirdSQL version 1.5.2

About the hardware/database settings used:

All databases were installed with default settings.
Tests were run on 1.6GHz Sempron with 1GB of ram and 7200rpm SATA disk running Windows 2000 + SP4 with all updates applied.

Test 1: 1000 INSERTs

CREATE TABLE t1(a INTEGER, b INTEGER, c VARCHAR(100));INSERT INTO t1 VALUES(1,13153,’thirteen thousand one hundred fifty three’);INSERT INTO t1 VALUES(2,75560,’seventy five thousand five hundred sixty’);… 995 lines omittedINSERT INTO t1 VALUES(998,66289,’sixty six thousand two hundred eighty nine’);INSERT INTO t1 VALUES(999,24322,’twenty four thousand three hundred twenty two’);INSERT INTO t1 VALUES(1000,94142,’ninety four thousand one hundred forty two’);

SQLite 3.3.3 (sync):
3.823
SQLite 3.3.3 (nosync):
1.668
SQLite 2.8.17 (sync):
4.245
SQLite 2.8.17 (nosync):
1.743
PostgreSQL 8.1.2:
4.922
MySQL 5.0.18 (sync):
2.647
MySQL 5.0.18 (nosync):
0.329
FirebirdSQL 1.5.2:
0.320

Test 2: 25000 INSERTs in a transaction

BEGIN;CREATE TABLE t2(a INTEGER, b INTEGER, c VARCHAR(100));INSERT INTO t2 VALUES(1,298361,’two hundred ninety eight thousand three hundred sixty one’);… 24997 lines omittedINSERT INTO t2 VALUES(24999,447847,’four hundred forty seven thousand eight hundred forty seven’);INSERT INTO t2 VALUES(25000,473330,’four hundred seventy three thousand three hundred thirty’);COMMIT;

SQLite 3.3.3 (sync):
0.764
SQLite 3.3.3 (nosync):
0.748
SQLite 2.8.17 (sync):
0.698
SQLite 2.8.17 (nosync):
0.663
PostgreSQL 8.1.2:
16.454
MySQL 5.0.18 (sync):
7.833
MySQL 5.0.18 (nosync):
7.038
FirebirdSQL 1.5.2:
4.280

Test 3: 25000 INSERTs into an indexed table

BEGIN;CREATE TABLE t3(a INTEGER, b INTEGER, c VARCHAR(100));CREATE INDEX i3 ON t3(c);… 24998 lines omittedINSERT INTO t3 VALUES(24999,442549,’four hundred forty two thousand five hundred forty nine’);INSERT INTO t3 VALUES(25000,423958,’four hundred twenty three thousand nine hundred fifty eight’);COMMIT;

SQLite 3.3.3 (sync):
1.778
SQLite 3.3.3 (nosync):
1.832
SQLite 2.8.17 (sync):
1.526
SQLite 2.8.17 (nosync):
1.364
PostgreSQL 8.1.2:
19.236
MySQL 5.0.18 (sync):
11.524
MySQL 5.0.18 (nosync):
12.427
FirebirdSQL 1.5.2:
6.351

Test 4: 100 SELECTs without an index

SELECT count(*), avg(b) FROM t2 WHERE b>=0 AND b=100 AND b=200 AND b=9700 AND b=9800 AND b=9900 AND bTest 5: 100 SELECTs on a string comparison

SELECT count(*), avg(b) FROM t2 WHERE c LIKE ‘%one%’;SELECT count(*), avg(b) FROM t2 WHERE c LIKE ‘%two%’;SELECT count(*), avg(b) FROM t2 WHERE c LIKE ‘%three%’;… 94 lines omittedSELECT count(*), avg(b) FROM t2 WHERE c LIKE ‘%ninety eight%’;SELECT count(*), avg(b) FROM t2 WHERE c LIKE ‘%ninety nine%’;SELECT count(*), avg(b) FROM t2 WHERE c LIKE ‘%one hundred%’;

SQLite 3.3.3 (sync):
4.853
SQLite 3.3.3 (nosync):
4.868
SQLite 2.8.17 (sync):
4.511
SQLite 2.8.17 (nosync):
4.500
PostgreSQL 8.1.2:
6.565
MySQL 5.0.18 (sync):
3.424
MySQL 5.0.18 (nosync):
2.090
FirebirdSQL 1.5.2:
5.803

Test 6: INNER JOIN without an index

SELECT t1.a FROM t1 INNER JOIN t2 ON t1.b=t2.b;
SQLite 3.3.3 (sync):
14.473
SQLite 3.3.3 (nosync):
14.445
SQLite 2.8.17 (sync):
47.776
SQLite 2.8.17 (nosync):
47.750
PostgreSQL 8.1.2:
0.176
MySQL 5.0.18 (sync):
3.421
MySQL 5.0.18 (nosync):
3.443
FirebirdSQL 1.5.2:
0.141

Test 7: Creating an index

CREATE INDEX i2a ON t2(a);CREATE INDEX i2b ON t2(b);
SQLite 3.3.3 (sync):
0.552
SQLite 3.3.3 (nosync):
0.526
SQLite 2.8.17 (sync):
0.650
SQLite 2.8.17 (nosync):
0.605
PostgreSQL 8.1.2:
0.276
MySQL 5.0.18 (sync):
1.159
MySQL 5.0.18 (nosync):
0.275
FirebirdSQL 1.5.2:
0.264

Test 8: 5000 SELECTs with an index

SELECT count(*), avg(b) FROM t2 WHERE b>=0 AND b=100 AND b=200 AND b=499700 AND b=499800 AND b=499900 AND bTest 9: 1000 UPDATEs without an index

BEGIN;UPDATE t1 SET b=b*2 WHERE a>=0 AND a=10 AND a=9980 AND a=9990 AND aTest 10: 25000 UPDATEs with an index

BEGIN;UPDATE t2 SET b=271822 WHERE a=1;UPDATE t2 SET b=28304 WHERE a=2;… 24996 lines omittedUPDATE t2 SET b=442549 WHERE a=24999;UPDATE t2 SET b=423958 WHERE a=25000;COMMIT;

SQLite 3.3.3 (sync):
1.883
SQLite 3.3.3 (nosync):
1.894
SQLite 2.8.17 (sync):
1.994
SQLite 2.8.17 (nosync):
1.973
PostgreSQL 8.1.2:
23.933
MySQL 5.0.18 (sync):
16.348
MySQL 5.0.18 (nosync):
17.383
FirebirdSQL 1.5.2:
15.542

Test 9: 1000 UPDATEs without an index

BEGIN;UPDATE t1 SET b=b*2 WHERE a>=0 AND a=10 AND a=9980 AND a=9990 AND aTest 10: 25000 UPDATEs with an index

BEGIN;UPDATE t2 SET b=271822 WHERE a=1;UPDATE t2 SET b=28304 WHERE a=2;… 24996 lines omittedUPDATE t2 SET b=442549 WHERE a=24999;UPDATE t2 SET b=423958 WHERE a=25000;COMMIT;

SQLite 3.3.3 (sync):
3.153
SQLite 3.3.3 (nosync):
3.088
SQLite 2.8.17 (sync):
3.993
SQLite 2.8.17 (nosync):
3.983
PostgreSQL 8.1.2:
5.740
MySQL 5.0.18 (sync):
2.718
MySQL 5.0.18 (nosync):
1.641
FirebirdSQL 1.5.2:
2.976

If you want to see some more information on the above and 10 more tests, you can go Here

Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • Digg
  • del.icio.us
  • DZone
  • Slashdot
  • StumbleUpon
  • Technorati
6 comments

Next Page »