Archive for the 'tech' Category
why do open source projects have so many bugs?
By Justin Silverton
In a recent article:
“In his first “meet the press” session as ICT Minister, Professor Sitthichai Pokai-udom put forth his radical vision for 3G in Thailand, condemned open source for turning out buggy, useless software and promised to make the civil servants in the ICT Ministry proud of their organisation once again.
Speaking at the IT Press Club, Minister Sitthichai said that he was looking for an investment of around 20 billion baht for Thai Mobile and between 10 and 18 billion for CAT in order to complete the roll-out of next generation communication networks for the two former state enterprises.
Sitthichai said on detailed questioning that Thai Mobile would upgrade its network and operate a WCDMA 3G network while CAT Telecom would roll out a CDMA2000 EV-DO network.
He said that rather than using public money to compete with the private sector, he expected today’s private mobile operators to gladly become partners to CAT and TOT and effectively lease their networks and focus on the marketing aspect. He reasoned that none of the incumbents would want to invest in a 3G network as 4G would come along in five years and render today’s investments obsolete.”
While I don’t completely agree with the stance that all open source applications are bug-ridden and useless, he does make a good point. I have used many open source applications over the years and many were either bug-ridden, barely functional, or they lacked basic functionali.
Open source sounds like a great concept. The idea is that since the code is open, there is a possibility for more programmers to see the code and create new functionality or fix bugs. This does happen to some degree, but because of the overwhelming number of contributers, many of the projects are subjected to in-fighting, politics, and eventual destruction.
While this is a part of any project, there needs to be more organization for a project to be successful. This involves assigning leaders and only accepting code into the main codebase that meets a certain standard. Many of the larger and more popular projects are using this method and have less bugs as a result (and are around much longer).
If more open source projects followed better project management practices, We would see much better projects an it would benefit both the users and the programming comunity.
Top 5 freelance websites
By Justin Silverton
The following are the top 5 websites that I have found for freelance work. This is for mostly programming/development, which includes PHP and many other languages.
Rentacoder is good place to find small to mid-sized projects for earning extra money. It also offers a rating system which can hurt or help you depending on your work.
Odesk is a little bit different than the rest of the sites listed here. Instead of bidding on projects, you bid on hours. Employers can list the amount of hours/week and # of weeks they need out of a potential freelancer.
Craigslist is great because there is no barrier to entry for either the employer or the potential freelancer/employee (it is also among the top sites on the Internet). This can also be a detriment because anyone with a computer can post a job listing.
2) http://programmermeetdesigner.com/
A unique website that provides an opportunity for programmers to meet and work with designers.
Guru.com is a clean and professional website that has lots of large-scale projects. It also has a fairly high barrier to entry for the employers, which usually means better paying jobs.
How efficient is your favorite language?
By Justin Silverton
How does you favorite language compare? Below is the results of a series of tests performed on various programming languages. The weights of each specific test are listed in the second chart below.
|
 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Weights used for the scorecard | |||
|---|---|---|---|
| Test | Weight | Test | Weight |
| Ackermann’s Function | 1 | Array Access | 3 |
| Count Lines/Words/Chars | 3 | Echo Client/Server | 5 |
| Exception Mechanisms | 1 | Fibonacci Numbers | 2 |
| Hash (Associative Array) Access | 1 | Hashes, Part II | 4 |
| Heapsort | 4 | Hello World | 1 |
| List Operations | 3 | Matrix Multiplication | 3 |
| Method Calls | 5 | Nested Loops | 4 |
| Object Instantiation | 5 | Producer/Consumer Threads | 1 |
| Random Number Generator | 3 | Regular Expression Matching | 4 |
| Reverse a File | 4 | Sieve of Erathostenes | 4 |
| Spell Checker | 4 | Statistical Moments | 2 |
| String Concatenation | 2 | Sum a Column of Integers | 3 |
| Word Frequency Count | 5 | ||
(Chart reference: http://dada.perl.it/)
Flickr.com - PHP/mysql case study
Introduction
Carl Henderson from Flickr.com, a very popular photo blogging service has released a pdf (not sure exactly when this was actually released) detailing the issues they faced with having a high-traffic website.
original PDF can be downloaded here
some interesting points taken from this pdf are below.
Classes, libraries, and systems used
1) smarty for templating
2) PEAR and XML for Email parsing
3) perl for controlling
4) imagemagick for image processing
5) mysql (4.0/innoDb)
6) java, for node service
7) apache 2 and redhat linux
8) 60,000 lines of PHP code
9) 60,000 lines of templates
10) 70 custom smarty functions/modifiers
11) 25,000 DB transactions/second at peak
12) 1000 pages per second at peak
unicode support
1) UTF-8 pages
2) CJKV support
Tips: don’t use HtmlEntities(). Also, Javascript has patchy Unicode Support
Why php was used
1) Everything can be stored in the database, including smarty cache
2) a “shared nothing” approach (as long as php sessions were not used)
Mysql usage
Select’s: 44,220,588
Insert’s: 1,349,234
update’s: 1,755,503
delete’s: 318,439
13 select’s per Insert, Update, and delete
Tips: many of the tables that needed to be full-text searched were de-normalized. This does waste space, but because it allowed for little or no joins, it made searching much faster.
10 database speed tests
By Justin Silverton
I came across the following 10 benchmark tests covering:
SQLite version 3.3.3
SQLite version 3.3.3
SQLite version 2.8.17
SQLite version 2.8.17
PostgreSQL version 8.1.2
MySQL version 5.0.18
FirebirdSQL version 1.5.2
About the hardware/database settings used:
All databases were installed with default settings.
Tests were run on 1.6GHz Sempron with 1GB of ram and 7200rpm SATA disk running Windows 2000 + SP4 with all updates applied.
Test 1: 1000 INSERTs
CREATE TABLE t1(a INTEGER, b INTEGER, c VARCHAR(100));INSERT INTO t1 VALUES(1,13153,’thirteen thousand one hundred fifty three’);INSERT INTO t1 VALUES(2,75560,’seventy five thousand five hundred sixty’);… 995 lines omittedINSERT INTO t1 VALUES(998,66289,’sixty six thousand two hundred eighty nine’);INSERT INTO t1 VALUES(999,24322,’twenty four thousand three hundred twenty two’);INSERT INTO t1 VALUES(1000,94142,’ninety four thousand one hundred forty two’);
SQLite 3.3.3 (sync):
3.823
SQLite 3.3.3 (nosync):
1.668
SQLite 2.8.17 (sync):
4.245
SQLite 2.8.17 (nosync):
1.743
PostgreSQL 8.1.2:
4.922
MySQL 5.0.18 (sync):
2.647
MySQL 5.0.18 (nosync):
0.329
FirebirdSQL 1.5.2:
0.320
Test 2: 25000 INSERTs in a transaction
BEGIN;CREATE TABLE t2(a INTEGER, b INTEGER, c VARCHAR(100));INSERT INTO t2 VALUES(1,298361,’two hundred ninety eight thousand three hundred sixty one’);… 24997 lines omittedINSERT INTO t2 VALUES(24999,447847,’four hundred forty seven thousand eight hundred forty seven’);INSERT INTO t2 VALUES(25000,473330,’four hundred seventy three thousand three hundred thirty’);COMMIT;
SQLite 3.3.3 (sync):
0.764
SQLite 3.3.3 (nosync):
0.748
SQLite 2.8.17 (sync):
0.698
SQLite 2.8.17 (nosync):
0.663
PostgreSQL 8.1.2:
16.454
MySQL 5.0.18 (sync):
7.833
MySQL 5.0.18 (nosync):
7.038
FirebirdSQL 1.5.2:
4.280
Test 3: 25000 INSERTs into an indexed table
BEGIN;CREATE TABLE t3(a INTEGER, b INTEGER, c VARCHAR(100));CREATE INDEX i3 ON t3(c);… 24998 lines omittedINSERT INTO t3 VALUES(24999,442549,’four hundred forty two thousand five hundred forty nine’);INSERT INTO t3 VALUES(25000,423958,’four hundred twenty three thousand nine hundred fifty eight’);COMMIT;
SQLite 3.3.3 (sync):
1.778
SQLite 3.3.3 (nosync):
1.832
SQLite 2.8.17 (sync):
1.526
SQLite 2.8.17 (nosync):
1.364
PostgreSQL 8.1.2:
19.236
MySQL 5.0.18 (sync):
11.524
MySQL 5.0.18 (nosync):
12.427
FirebirdSQL 1.5.2:
6.351
Test 4: 100 SELECTs without an index
SELECT count(*), avg(b) FROM t2 WHERE b>=0 AND b=100 AND b=200 AND b=9700 AND b=9800 AND b=9900 AND bTest 5: 100 SELECTs on a string comparison
SELECT count(*), avg(b) FROM t2 WHERE c LIKE ‘%one%’;SELECT count(*), avg(b) FROM t2 WHERE c LIKE ‘%two%’;SELECT count(*), avg(b) FROM t2 WHERE c LIKE ‘%three%’;… 94 lines omittedSELECT count(*), avg(b) FROM t2 WHERE c LIKE ‘%ninety eight%’;SELECT count(*), avg(b) FROM t2 WHERE c LIKE ‘%ninety nine%’;SELECT count(*), avg(b) FROM t2 WHERE c LIKE ‘%one hundred%’;
SQLite 3.3.3 (sync):
4.853
SQLite 3.3.3 (nosync):
4.868
SQLite 2.8.17 (sync):
4.511
SQLite 2.8.17 (nosync):
4.500
PostgreSQL 8.1.2:
6.565
MySQL 5.0.18 (sync):
3.424
MySQL 5.0.18 (nosync):
2.090
FirebirdSQL 1.5.2:
5.803
Test 6: INNER JOIN without an index
SELECT t1.a FROM t1 INNER JOIN t2 ON t1.b=t2.b;
SQLite 3.3.3 (sync):
14.473
SQLite 3.3.3 (nosync):
14.445
SQLite 2.8.17 (sync):
47.776
SQLite 2.8.17 (nosync):
47.750
PostgreSQL 8.1.2:
0.176
MySQL 5.0.18 (sync):
3.421
MySQL 5.0.18 (nosync):
3.443
FirebirdSQL 1.5.2:
0.141
Test 7: Creating an index
CREATE INDEX i2a ON t2(a);CREATE INDEX i2b ON t2(b);
SQLite 3.3.3 (sync):
0.552
SQLite 3.3.3 (nosync):
0.526
SQLite 2.8.17 (sync):
0.650
SQLite 2.8.17 (nosync):
0.605
PostgreSQL 8.1.2:
0.276
MySQL 5.0.18 (sync):
1.159
MySQL 5.0.18 (nosync):
0.275
FirebirdSQL 1.5.2:
0.264
Test 8: 5000 SELECTs with an index
SELECT count(*), avg(b) FROM t2 WHERE b>=0 AND b=100 AND b=200 AND b=499700 AND b=499800 AND b=499900 AND bTest 9: 1000 UPDATEs without an index
BEGIN;UPDATE t1 SET b=b*2 WHERE a>=0 AND a=10 AND a=9980 AND a=9990 AND aTest 10: 25000 UPDATEs with an index
BEGIN;UPDATE t2 SET b=271822 WHERE a=1;UPDATE t2 SET b=28304 WHERE a=2;… 24996 lines omittedUPDATE t2 SET b=442549 WHERE a=24999;UPDATE t2 SET b=423958 WHERE a=25000;COMMIT;
SQLite 3.3.3 (sync):
1.883
SQLite 3.3.3 (nosync):
1.894
SQLite 2.8.17 (sync):
1.994
SQLite 2.8.17 (nosync):
1.973
PostgreSQL 8.1.2:
23.933
MySQL 5.0.18 (sync):
16.348
MySQL 5.0.18 (nosync):
17.383
FirebirdSQL 1.5.2:
15.542
Test 9: 1000 UPDATEs without an index
BEGIN;UPDATE t1 SET b=b*2 WHERE a>=0 AND a=10 AND a=9980 AND a=9990 AND aTest 10: 25000 UPDATEs with an index
BEGIN;UPDATE t2 SET b=271822 WHERE a=1;UPDATE t2 SET b=28304 WHERE a=2;… 24996 lines omittedUPDATE t2 SET b=442549 WHERE a=24999;UPDATE t2 SET b=423958 WHERE a=25000;COMMIT;
SQLite 3.3.3 (sync):
3.153
SQLite 3.3.3 (nosync):
3.088
SQLite 2.8.17 (sync):
3.993
SQLite 2.8.17 (nosync):
3.983
PostgreSQL 8.1.2:
5.740
MySQL 5.0.18 (sync):
2.718
MySQL 5.0.18 (nosync):
1.641
FirebirdSQL 1.5.2:
2.976
If you want to see some more information on the above and 10 more tests, you can go Here