Coding is often just a small part of software development. There are lots of other tasks to attend to. Here are some tools to help with all aspects of the job.
Not all sites proffer slick RESTful interfaces and XML feeds. In those cases, collecting data requires some good, old-fashioned scraping. This week, let’s look at some of the tools and techniques required to scrape a site.
Rails is a productive environment — once you install gems, plugins, and initializers, configure routes, generate code, migrate databases, and tune your environments. Rails templates can automate all of that busy work.
One day, we won’t have to worry about browser eccentricities. (I have a dream.) Until then, tools like Browserlab make the horrible realities of Web development tolerable.
Webby is ideal for a content archive, a small business or governmental website, and even small catalogs that link to PayPal for commerce. Here, learn how to use Webby to create a site fast.
Gearman enables a new level of software abstraction. With this lightweight infrastructure you can to outsource work to better-suited computers, run tasks in parallel, and combine code written in different computer languages.
Are Rails, CakePHP, Django, and Catalys too big for your taste but you still want the benefits of DRY programming? Maybe it’s time to get small with micro-frameworks.
Are the “Two Ps” (print statements and prayer) all that comprise your debugging strategy? It’s time you were introduced to the Interactive Ruby Debugger.
Despite some of the concerns floating around the MySQL Conference this week, there’s some good news coming out of the event. The MySQL developers are returned to a “release early, release often” schedule and the pending 5.4 release has a number of features worth keeping an eye on.
Protecting the sanity of Rails developers is job #1! Reduce new project setup time and speed the installation of essential gems and plug-ins with new application templates.
Compiled programming languages and scripting languages each have unique advantages, but what if you could use both to create rich applications? Lua is an embeddable scripting language that is small, fast, and very powerful. Before you create yet another configuration file or resource format (and yet another parser to accompany it), try Lua.
Have a penguinista in your life? Need the perfect holiday gift? Look no further! Here’s a parcel of presents that’ll please even the pickiest geek. Best of all, none will put your eye out.
A recent poll of European Union local governments shows increasing adoption of free and open source software. Indeed, uptake of the Penguin is happening everywhere, albeit quietly.
Change owner, mode, and group of an entire directory all at once using chown. For example, chown -R a+rw joe:dev dir makes dir and everything in dir world-readable and -writeable, and sets owner to joe and group to dev.
If you want to know what process(es) are running a certain application, use pidof. For example, pidof /bin/zsh yields all process ID’s running /bin/zsh.
More than the protocols, backbones, and servers that underlie the Internet, open source licenses have
facilitated the exchange of source code and ideas. Here’s a look at academic and reciprocal licenses and how to use them.
Once again, Massachusetts is in the middle of a revolt. And while the state may not go so far as dumping copies of Word over the side of the Eleanor, like the original Boston Tea Party, it’s the intention that counts.
With MySQL 5 now widely available, the MySQL founder weighs in on the competition, the future of the MySQL database, and how to manage contributed code. Here’s ten questions with the co-founder and CTO of MySQL AB.
The phrase “open source license” refers to a large number of agreements that license the
copyrights inherent in software widely, fairly, and with the fewest restrictions possible. This article —
the first of two — describes the tenets of copyright and explains the intents of an open source license.
The second article in this series explores individual licenses, such as the GNU Public License and the
Apache License.
Tired of CVS? Try darcs. Creating a repository with darcs is as easy as cd ~/myproject; darcs init; darcs add -r *. Use apt-get install darcs to install or see the darcs wiki for source, tools, and more info.
If you’re not careful, mv can overwrite existing files. Move files wisely with mv –backup=existing -v. To save typing, set VERSION_CONTROL to existing.
MySQL AB, developer of the world’s most popular open source database, today announced the general availability of MySQL 5.0, the most sigificant product upgrade in the company’s ten-year history. Starting today, MySQL 5.0 can be downloaded under the open source GPL license at http://dev.mysql.com.
You can quickly create new user accounts or update existing accounts with the newusers command. Create a list of accounts in a passwd-like file and run the command newusers file.
Build your own view of file metadata (permissions, file name, file size, and so on) with the stat command. Use -c (or –format=) to display the fields you want, in the order you want — perfect for processing with scripts!
Avoid logging in as root. Instead, use sudo to run individual commands as the superuser. As Spider-man says, “With great power comes great responsibility.”
Need to find or kill a process? Use the utilities pgrep and pkill. For example, the command pkill -INT -u root sshd sends the INT signal to the sshd process owned by root.
If you have a comma-delimited file, use cut -s -d, -f n file to extract the nth field from each line of file. The -s option skips lines without the delimiter.
View and process a gzip-compressed file (such as an archived log file) without expanding it using zcat, zdiff, zmore, zless, zcmp, zgrep, zegrep, and zfgrep.
wget is an invaluable utility. Use it from the command-line or a shell script to download a web page, an entire web site, a tarball, or just about any http: or ftp: URL. wget can also help debug HTTP and FTP connection problems.
Which would you prefer? Chocolate day in and day out or a new flavor every day of the month? Much maligned, the smorgasbord of Linux distributions is really quite delicious.
If you want to watch a log file, use tail -F. The -F option prints new content as it’s appended (like -f), but also re-opens a file if it’s created anew, as occurs when a log file is rotated.
You don’t have to know Perl to use Perl. The command-line perl -p -i -e ’s/oldstring/newstring/g’ * replaces all occurrences of oldstring with newstring in all files in the current directory and makes backup copies of the original files.
The GNOME project has released version 2.12, “the latest version of the popular, multi-platform free desktop environment, providing all the tools a user needs for everyday work, and a platform for developers to write new software.”
Need to find a system configuration file but only remember a part of its name? Use locate. It’s faster than running find / … and much simpler to use. (Run updatedb often to keep the locate file database up-to-date.)
If you use pushd and popd to maintain a directory stack, use ~n on the command-line to reference the nth element of the stack with (where n is 0, 1, 2, and so on).
Poor, poor Microsoft. It has gobs of cash, a small army of employees, and the greatest monopoly ever and it’s still boring as hell. Microsofties, bring the flames!
Last month, I expressed boredom with the personal computer. Beyond gigahertz, gigabytes, and wireless, I complained, personal computers sold today look and feel a lot like those sold ten years ago.
Every so often, something truly magical appears and changes the universe forever. Recent marvels like wireless and digital content are just two examples that come to mind. Each changed our expectations of what’s possible.
With the recent acquisitions of Ximian and SuSE, Novell Chief Executive Officer Jack Messman has transformed a marginalized company into a mover-and-shaker. Look out, Red Hat - and Microsoft - Novell has big plans for Linux.
Back in 1985, in my last year of graduate school, I was lucky enough to get a “real” office, including a door, a lock and key, a telephone, and a shiny, new Sun 3/110 workstation. After timesharing with the general population on VAX 11s for three years, the Sun machine was a godsend: a bitmapped screen, shell windows, and all the processing power of a 16 MHz 68020. (Man, those were the days!)
If you ask a Perl aficionado a programming question, he or she is apt to reply “There’s more than one way to do it.” That response isn’t trite or boastful: Perl offers many different ways to solve the same problem.
In the near future, the city of Cincinnati, Ohio will become the first municipality in the country to receive broadband over power lines (BPL). The service, which promises at least one megabit per second download and upload speeds, can be accessed via any electrical socket using an adapter that closely resembles a large cell-phone charger. The technology is proven, the Federal Communications Commission is endorsing it, and unlike DSL or wireless, BPL can go anywhere that power goes. And BPL is cheap: the basic one mbps service is priced at $29.95 per month.
I am happy to report that I’ve finally completed Madame Pearl’s “Clairvoyance for Fun and Profit” correspondence course. For the modest tuition of $1,000, I’ve received hours of expert instruction via webcast, a sheepskin for my office (soon to be parlor), a deck of tarot cards autographed by Madame Pearl herself, and a shiny new iBall, a wireless crystal ball that peers into the future.
As this month’s linuxdlsazine goes to press, the mainstream media and the technorati (the geek paparazzi) are widely reporting on the Open Source Development Labs’ creation of a Linux legal defense fund. Briefly (and you can read more details in our “Report from the Front” on page 8), OSDL’s fund will subsidize the legal expenses of any Linux end-user that’s threatened by broad intellectual property claims, such as SCO’s assertion that code in the Linux kernel infringes its copyrights. OSDL plans to raise $10 million for the fund; the fund’s been seeded with $3 million by IBM, Intel, and others.
Like the heart and lungs, accounts payable and accounts receivable keep a company pumping. Money goes out; raw materials come in. Products and services go out; money comes in. If more money comes in than goes out, the company prospers. At least that’s the theory — and the goal.
If someone asked you to describe Linux, what would you say? If you’re pedantic, perhaps you’d say, “Linux is an operating system kernel.” If you’re practical, maybe you’d explain, “Linux is an operating system where you can inspect and change any of the components.” Or, if you’re evangelical, perhaps you’d proffer, “It’s like Windows, only there’s no blue screen of death, and it’s free!”
While email may now and forever be the Internet’s ultimate “killer app,” there was a time when BBSs or bulletin board systems — electronic equivalents of the neighborhood message board — ruled cyberspace. Back then, email systems were sparsely connected, and sending email often required you to remember or discover transmission paths. (Perhaps some of you remember specifying routing instructions in email addresses with %, !s, and @.) Instead, computer hobbyists and nascent Internet users dialed-up services like Usenet and The Well to meet, debate, discuss, and yes, flame.
After fifteen years, I’m paying for it. Sure, I could be patient, and maybe get it for free later, but I want it now — and am willing to fork over cash to get it. So, with a satisfying click of the mouse, it’s mine.
I woke up today with recall on my mind. No, not the drama playing out in California these days — the recall I have in mind has much more significance. Hear me out.
While Red Hat’s complaint and IBM’s countersuit provided fan-pleasing left and right hooks to SCO on the first morning of the most recent LinuxWorld Expo, the most important story of the show lay underneath the flurry of press releases, swagger, and bluster: for the first time ever, the vaunted GNU Public License (GPL), the constitution of much of the free software world may be put to a legal test.
Let’s say that you’re a system administrator responsible for a large set of computer systems — everything from Linux servers to Macintosh desktops. Like most system administrators, you’re very busy, tracking down problems, installing new software, monitoring systems, and helping end-users. Sounds pretty typical, huh?
2.6 is coming, and Andrew Morton, a modest, humble, approachable, and very capable system software developer is leading the charge. Hand-picked by Linus Torvalds for the task, Morton talks about the next production kernel, the kernel development process, and what would happen if SCO won its case against IBM.
This past month, I received much more email than usual. The reason? The Microsoft ad and the Microsoft CD-ROM included with the August issue of linuxdlsazine.
As this month’s issue of linuxdlsazine goes to press, the SCO fear and uncertainty dreadnought continues to strip-mine the landscape of Linux. Each day of the last week brought more SCO allegations and threats. SCO even took Linus’ approach to developing the kernel to task, practically calling the massively collaborative model near-willful infringement. But while dung spewed from Utah, IBM shrugged it all off. In fact, Big Blue looks Very Cool Blue so far.
While the “P” in “LAMP” most often brings to mind Perl or PHP, the other “P,” Python, is no third-wheel. Indeed, with powerful language structures that support objects, dynamic data types, and dynamic typing, and a rich trove of core and contributed programming libraries, the variety of applications that can be built with Python is quite astounding. The Python Users page (found online at http://www.python.org/psa/Users.html) lists just a small sampling: there’s a content management system framework (Zope); a Quake level editor (Quake Army Knife, or “Quark”); and an entire web site — PalmStation.com — written solely in Python.
The Internet was originally conceived to improve communication between far-flung researchers. Today, of course, the Internet can be used by anyone, virtually anywhere, to send and receive information of all kinds. Email, newsgroups, web sites, and more recently, blogs, and RSS feeds are all methods to share information.
Back in 1989, I discovered the X Window System, and almost immediately, I fell in love with it. X was fun to play with and a blast to program for. In fact, I was so enamored of graphical user interfaces and the ins-and-outs of windowing systems, I spent the next three years or so working on little else but X. I even had the privilege of participating as a member of the X Consortium, shuttling regularly back and forth between coasts for X technical conferences and hands-on “interop” sessions. Indeed, to this day, I trace my eventual career in consumer software and computer game development back to the day when I launched X on my Sun 3/60 workstation.
In the past several months, a good number of corporate, government, academic, and research institutions — Pixar, the Lawrence Livermore National Lab (LLNL), the Centers for Disease Control (CDC), Shell E&P, and others — have announced the installation of substantial, high performance Linux computing clusters. In the case of LLNL, for example, the largest of its three new clusters (built by Linux Networx) is composed of 252 Pentium 4 processors, capable of a theoretical peak of 857 gigaflops, making it one of the fastest clusters ever built. In the case of Pixar, a 1,024-processor blade cluster (using 2.8 GHz Xeons) from RackSaver is replacing the company’s existing Sparc-based render farm.
While email and instant messaging are great ways to stay in touch — and are nearly ubiquitous — those “killer apps” are just a little dull. After all, why send email to your brother when you can just as easily reach out and “frag” him?
Electronic mail is both a blessing and a curse. On the one hand, email is so convenient: You can send email at any time from almost any computer, and email is great for staying connected to colleagues, friends, and family members, especially if you’re separated by time zones or great distances. Moreover, email costs nothing and is easily shared with groups of any size. On the other hand, email can be overwhelming. Today, it’s not unusual to receive hundreds of email messages a day. Sure, reading and replying to all of that correspondence has to be done, but filtering and filing all of that email (not to mention fighting spam) can be downright daunting and time-consuming.
As this issue of linuxdlsazine goes to print, the biggest story in the Linux world is SCO’s “not less than” $1 billion law suit against IBM. News of the suit spread quickly over Web, blogs, and wire. InfoWorld ran the headline, “SCO Sues IBM Over Linux,” while c|net wrote, “SCO Sues Big Blue over Unix, Linux.”
With over two million downloads in 2002, JBoss is arguably the de facto standard for deploying Java-based Web applications. With its advanced features, microkernel architecture, full implementation of the J2EE stack, and an unbeatable price (it’s free, available as source code released under the Lesser Gnu Public License), JBoss — like Linux and Apache — has been widely adopted by developers and corporate IT departments.
I’ve just returned from Linux World in New York, and besides a mild case of jet lag and frostbite, I’m back in the swing of things, trying to herd 50,000 words into something resembling a magazine. (Drop me a line and rate my wrangling skills.) And, in addition to recovering from my trip to the Big City, I’m also trying to form an opinion on this most recent Linux World. Everyone asks, “How was the show?” and, to be honest, I’m not sure what to say.
This month’s issue of linuxdlsazine is our second annual “Java” issue. As we did in March of last year, we’ve set aside the features of the magazine to explore new and emerging Java technology that you’re likely to hear more about — and if you’re a Java programmer, probably code with — in the months ahead.
In 2001, revenue from the sales of video games exceeded $9 billion, a new record. By the end of October 2002, sales of video games had already surpassed $6 billion, and was expected to eclipse $10 billion by year-end, setting yet another record (a final revenue tally for 2002 was not available as we went to press).
Web Services have unleashed a raft of technologies, but are any effective? A number of developers say no and suggest that the Web already offers everything you need to build a great Web Service. And more.
One of the many amazing and marvelous facets of the Open Source movement is its volunteer spirit. All across the Internet — on the Web, and in newsgroups, blogs, and CVS trees — people from all walks of life, spanning all parts of the globe, contribute their time, energy, and expertise to the cause. It’s quite remarkable, and even though I’m exposed to Linux projects every day, I still find the whole Open Source process fascinating.
If you live in the San Francisco Bay Area long enough, your view of the world gets a little warped. “Hot weather” is anything above 75 degrees, “rain” is a four-letter word, and the East Coast might as well be a part of the European Union.
As we wrap us this issue of linuxdlsazine and wrap up the year, I keep asking myself a simple question: Is Linux better off now than it was a year ago?
Web services promise to integrate businesses, connect all kinds of devices, and rally the computer industry around standards. You’ve heard the hype — here’s what the hubbub’s all about.
Java’s string processing capabilities are far better than those found in C, C++, or Visual Basic, but aren’t quite as powerful as, say, Perl’s. While the Java String class provides convenient methods such as indexOf(), lastIndexOf(), and startsWith(), you still have to write quite a bit of Java code to do something like break a colon-delimited record into a set of fields. Until now.
In March 2002, Sun Microsystems released Version 1.4 of its Java 2 Standard Edition (J2SE) software (http://java.sun.com/j2se/1.4). J2SE is the core of the Java platform. While other Java kits are specialized (e.g., J2ME is targeted at small devices and J2EE provides a range of server and e-commerce solutions), J2SE provides fundamental components that every Java programmer can use.