Channel Mike

Any time you depend upon data storage and manipilation on a daily basis for your business, or if you take your data seriously even if it's not mission-critical or used daily, you should be looking at database solutions. Spreadsheets can help you to analyze data. They can help you to create scenarios and plugin different assumptions. They can do a lot. But you need to build a safe, secure, comfortable house for your data; and a dedicated database is really the only way to do that.

On its face, a database is an incredibly simple thing. It's a collection of tables, each of which is a collection of records, each record having multiple fields. This is what your spreadsheet provides, too. A collection of records or rows where each record/row has several fields/columns. For this reason, if you're using a spreadsheet to house your data, you can move it over to a database pretty easily. This brings up a few questions, however. If that's all a database is, why not just use Excel? And why are they so expensive? And why are there so many options? And what are the differences between them? I don't intend this to be a full book, although it could grow to that scale quickly. But I intend to answer all of these questions to help you make a decision on where and how to house your data.

You've probably already accompanied me on a journey through spreadsheets and local database tools like Access and Filemaker. These are desktop applications. As we move into databases, we're moving away from applications that give you a single interface you can deal with visually (buttons, grids, pull-down menus, etc.) and into services that just run invisibly in the background, talking to separate applications that use their data storage capabilities. These interfaces use less buttons and menus, more programming and text commands. We're evolving from a single entity managing everything for you into a team of entities that each do what they're good at. Just like an individual contractor or consultant evolves from a one-man show into a team and eventually into a small business with different employees for different roles, Data management evolves from this single client application into multiple applications: a database back-end managing the data; combined with a front-end that manages the way you see and interact with it; And even middle-tier applications that verify and process data on its way in and out of the back-end database. This way, you can fire your front-end and get a new one without ever actually modifying your data on the back-end. You can even hire/build/buy a front-end for your accountant that filters and displays the data in a way an accountant would like; then hire/build/buy a front-end for your strategist or executive that filters the same data differently to provide a view of it that a big-picture person would appreciate. Your entry-level reps would get their own front-end with information they need to do their jobs laid out in an efficient and easy-to-understand manner for them. You could have a whole team of front-ends that each do a different job with the same exact data. This is one of the most powerful features of databases.

Lets start with Microsoft's offerings. Microsoft's databases are all called SQL Server, but come in different versions. The smallest, least powerful is SQL Server Compact Edition. It is designed to run on smart phones and mobile devices, but also runs on your laptop or desktop machine. It lacks much of the functionality of their larger offerings, notably a lack of multi-user functionality. But it takes up less than 2 megabytes of space on your machine (before data) if that's important to you. If you just need to house your data that you use all by yourself, this is a good option. Microsoft then moves up through SQL Server Express Edition, Workgroup Edition, Standard Edition, and Enterprise Edition. To make your choice, you need to look at your needs as an organization. The nice thing about Compact and Express is that they are both free. Just download the package (it will come with reference materials and tutorials) and start building. Unless you really need to fit the package on a smartphone or something, start with Express Edition. It will run on a desktop or a server so you can just install it on the same computer you're using to read this. If you run into something down the road that you need, you can upgrade to a more advanced edition and acquire those functions.

Some of the things you may run into as your needs grow are analysis and development tools that are included in the Enterprise Edition, but not others. You may need to house a lot of data. The Express Edition will only house 4GB of data and utilize 1GB of RAM on the system. This is an astronomical amount for housing business contacts and things you may have had in Excel, but won't cut it if you're running a credit card business and storing transaction data for thousands of customers every day. Analyzing populations of data for statistical or scientific analysis can also happen on a small scale, or a large scale. You need to determine the size of the data you're storing, then determine which tool you'll need.

As your needs grow, so will your costs. The Compact and Express editions are free, as mentioned before. But if you move up to the Workgroup edition because you are storing more than 4GB of data, you'll end up paying anywhere from $800 with restrictions on who can use the database up to $4000 to run a single server with no access restrictions. This all depends on the licensing model you choose. Moving up to Standard Edition to gain backup and mirroring functionality will cost from $2000 to $6000. And the full-scale Enterprise Edition with all features and all tools will cost from about $10,000 up to $25,000. Of course, if you need Enterprise Edition, you weren't storing your stuff in Excel. You've got some serious complexity and size to your data. For more detail, look at Microsoft's comparisons and licensing pages.

As we move away from Microsoft, Oracle is the industry leader in database technology. They have so many options and configurations and products that I can't possibly cover them here. But I can give the same rundown that I did with Microsoft's products. Most of Oracle's products sit atop their database and provide more complete solutions for your data needs than just an empty container. Their current version of an empty container, however, is Oracle 11g, which doesn't mean anything until we compare it with other versions. And this is relevant with Oracle because they offer older versions free of charge. You can download Oracle 10g Express Edition for free, but it has similar limitations to other free databases. Oracle 11g Standard Edition One starts at $750 for five users, the minimum configuration, up to $5000 for a single server with unlimited users. Standard Edition costs from $1,500 to $15,000. And Enterprise Edition runs from $4,000 to $40,000. You can then spend tens of thousands of dollars to add on capability to the Enterprise Edition. Many of the reasons for using Oracle databases would be to maintain compatibility with Oracle's middleware or applications that run on them. Oracle does have fast, robust, mature databases, however, and would provide a solid foundation for your own front-ends and analytical tools.

IBM has been in the database game for quite some time. They have historically specialized in mainframes and large data warehousing, and they have so many products and offerings on every level that they could put out a thick catalog with all of it. I will focus on DB2, their product line that competes most directly with Microsoft's and Oracle's database lines. They offer a free version, DB2 Express-C, and several more through Express 9, Workgroup 9, and Enterprise 9, all the way up to their mainframe giant, DB2 9 for z/OS. Their pricing model is based on "value units" so you pay more for a faster machine with more processors and less for an older slower machine. I have been unable to determine how much it would cost to run any but the free version from IBM. And the large companies I'm familiar with that use IBM have a full-time IBM employee on-site all the time as a liaison and for support. This screams at me that their offerings must be very expensive. But they have comparisons of licensing models and versions if you're interested.

Other options are those used by the open-source community. The most popular tools there are my favorite, mySQL, and Postgresql. Ingres is also available. MySQL version 5 is available free of charge, but comes with no support. But as the most common open-source database server on earth, you can find all the documentation you need if you're willing to look. You can spend from $600 to $5000 per server per year to get support, free timely updates, extra features, etc. PostgreSQL is free as well, and has a bunch of third-party add-ons that you can purchase. These open source solutions are both mature database engines that have been exhaustively tested. The only downside of the free software is that if something goes wrong, finding support is a more complicated problem than just calling your vendor. Both of these products do have commercial support, however. And you should be able to call them and get the help you need. I would strongly recommend open source to individuals hoping to learn about migrating to large databases. Building databases in mysql will give you everything you need to figure out how to build and design your data tables. It will allow you to see how to push and pull information with sql queries. And anything you build there will port pretty painlessly to any other relational database if you decide you need to change. And all of this is completely free.

There are quite a few players in the database market, even after acquisitions by Oracle and IBM of smaller vendors. Oracle, IBM, and Microsoft are the main players, followed by open source software, Sybase, and Teradata.

Once you've committed to doing the job right, you need to spend another $20 to $50 and get a decent database design book so that you can build your databases with efficient tables, proper data types, correct normalization, and possibilities for ease of expansion. Any book store ought to have dozens of books to choose from, including Amazon.com. You may want to look at both generic database books, as well as books specific to the database platform you've chosen. You'll also need a fair grasp of SQL once you get going. SQL is a language you can use to do about anything with your database. Any time you're putting data into your db or taking data out, you'll probably be using SQL or a tool that uses SQL behind the scenes. There's no shortage of books on SQL either.

Just about any sizable chunk of data stored today is in a relational database. Nearly every website uses at least one. Every company uses multiple databases whether they recognize it or not. If you work with data in any way, whether it's in an IT role or not, you're using databases. The more knowledge you have about them, the more success you'll have in getting what you want out of them. Enjoy the journey.