DBA of the Future
| Ed Dowgiallo | Howard Fosdick |
| Yuval Lirov and April Langer | |
| Tim Quinlan | Casey Young |
When we launched Database Programming & Design in 1987, few
could have predicted the incredible variety of systems that database administrators
(DBAs) must grapple with today. Ten years ago, the DBA role was still emerging;
in most corporations, it floated somewhere between application development
and systems management. And it was mostly part of the mainframe culture.
But very quickly, the DBA role developed into the tough job it is today:
managing an increasingly distributed, heterogeneous network of databases
under heavy demand as one of the corporation's most precious resources.
As the database industry continues to evolve, will the DBA's job become
automated-and will the DBA be cast off into oblivion like so much evolutionary
"dead weight"? Or will DBAs evolve along with the industry, taking
on new dimensions as they become caretakers of intelligent, complex, and
hence increasingly fallible systems? For this 10th-anniversary special feature,
a select panel of Database Programming & Design's most frequent
contributors-DBAs one and all-dare to predict. - The Editors
Ed Dowgiallo
Contributing Editor, DB Unplugged column (1995-1996)
For the database industry, the defining event of the next decade will be when the winner of the Oracle/Microsoft competition buys out the loser. The result of this merger will do for the 21st century economy what the creation of U.S. Steel did for the Industrial Revolution. The combined company, which will probably be named something like "OmniPresent Inc." ("GE Corp."is already taken), will produce every database product of any consequence.
Truly futile resistance will be offered by university spin-off companies started by researchers in knowledge representation and semantic database technology. However, Microsoft and Oracle will have been so distracted by their efforts toward mutual extermination early in the decade that by 2007, the trade press will be speculating on whether semantic databases will gradually eat at OmniPresent's marketshare, just as the relational startups gradually ate IBM's lunch in the 1980s.
At this point, the crystal ball gets fuzzy, but the press will most likely be wasting its time again. Bill Gates and Larry Ellison had the foresight to avoid this fate in the late 1990s by migrating pure relational technology to an object/relational hybrid, effectively neutering the current object-oriented database companies before they could gain significant market share. (The R&D has already been done by most of the major relational players and will roll out in the late 1990s. The first out of the gate? Oracle8.) By 2007, the fate of the semantic database companies will depend on whether Gates' and Ellison's successors have visions as clear as they had themselves.
For the troops in the trenches, there will no longer be a need to guess which products to become expert in using-OmniPresent will decide that for them, creating a feeling of comfort and stability forgotten since the 1970s when IBM was king. Intelligent database server technology will be mainstream, enabling servers to download edit and integrity checks to clients while centrally enforcing them. Network packet-level strategies for defeating this mechanism will be an area of growing interest to hackers and computer security specialists. As mobile computing becomes truly ubiquitous, databases that are not distributed will become rare. All client machines will have a local database system that must be synchronized periodically with a central server. This architecture-in conjunction with continued Internet expansion-will be the primary cause of a great telecommunications capacity crisis.
This crisis will originate from telephone companies, which will largely ignore growing demand in the hopes of forcing Congress to pay for their infrastructure upgrades. I won't venture a guess on what Congress will do, but the crisis will occur on several fronts. First of all, I predict that 10 years from now the country will still not be fully rewired with fiber-optic lines. In addition to wanting to move greater volumes of data of increasing complexity, individuals and businesses will become much more interested in wireless networking, fueling a universal demand for cellular Internet connections. For businesses, these connections will become a standard part of wide-area networking. (It's easy to imagine overnight delivery companies buying into this technology big-time as a replacement for their proprietary radio networks.) For individuals, the biggest factor involved will be the bandwidth made available for cellular Internet channels.
As for DBAs, the current trend toward specialized job functions will continue. As servers become more and more programmable, and as OmniPresent toils to make the database manager an integrated component of the operating system, the DBA profession as we know it will evolve into a set of more specialized jobs, some of which will involve non-DBA activities. For example, a class of specialists in distributed database/network capacity planning-" data pipeline architects," if you will-will design layout databases and networks to optimize data flow among master servers, applications, slave and replication servers, and client machines.
This group will most likely evolve from the current crop of tuning specialists, who will be displaced by expert systems that tune specific servers dynamically based on usage patterns. Fortunately, this trend will be more than offset by the increasing demand for people skilled in tuning groups of servers acting as distributed database systems. The quantum leap in complexity between the two areas will keep expert-system technology from progressing to this next step any time in the next 10 years.
A whole new niche will develop for people specializing in server programming. As client machines become the repository for ridiculously fancy yet unintuitive user interfaces-and the place where most of the nonfluff is downloaded from intelligent servers-professional programming will move to the back end. Specialized servers will proliferate as application servers for homegrown or customized packaged applications or for corporate services (as they're used for mail and fax services today).
In the chasm between these groups will be folks specializing in specific canned applications created by the never-ending industry aversion to training people who have a good general background on specific products. Newspaper ads will announce employment searches for "SAP v. N/Oracle v. M specific DBAs." These DBAs will be the key to customizing canned applications, which will become horrendously difficult to implement on one hand and flexible enough to gain popularity with users and cost-cutting managers on the other. In 2007, five years after the last Year 2000 project is completed two years late, these canned packages will finally send Cobol to its grave.
Physical DBAs will evolve into "system programmers" as the term is defined in the mainframe community. They will continue to be responsible for managing storage, integrity, and some types of tuning issues.
Finally, the question that's burning in all your minds: Ten years from now, will the main determinant of internal-development decision making in large corporations still be NIH ("Not Invented Here")?
Of course it will. Some things never change.
Ed Dowgiallo is an independent consultant specializing in relational
database technology. You can reach him via e-mail at [email protected].Back to top
Howard Fosdick
Most recent DBPD feature: "In Search of the Perfect Database Monitor," March 1996
DBAs were living in a totally different world in 1987. Is it possible to guess what their job will be like in 2007? The last 10 years demonstrate that predicting by extrapolation is hazardous-yet certainly less hazardous than not predicting at all.
One thing's for sure in the next decade: more hardware advances. Today's large databases (hundreds or even thousands of gigabytes) will fit on the midrange machines of the future. Desktop servers will hold enough data to run most businesses; even at the largest companies, a single desktop server will run many of the most significant applications. DBMSs will be the fundamental building blocks for handling this massive amount of data, much of it in new multimedia data types. On the high end, Very Large Databases (VLDBs) will take on a whole new meaning, facilitating new applications that are currently impossible.
For DBAs, these trends promise interesting jobs at the burgeoning number of small companies running their enterprises on desktop and midrange servers. And DBAs in large organizations will face the unique challenges and opportunities of building the vast new VLDBs of the future.
The current trend toward GUIs will continue and be accelerated by multimedia interfaces, making typical DBA tasks much easier. The continued explosion of the server population mandates ever-higher levels of server automation for distributed systems management, ultimately culminating in self-administered systems. Intelligent agents and self-monitoring systems will be required to control increasingly dispersed and diverse distributed databases.
These developments will whipsaw DBAs. On one hand, the demand for DBA services will be undercut as jobs are increasingly de-skilled and automated, with centralized control ever more possible for widely distributed systems. On the other hand, the geometric explosion in the number and functionality of machines ensures that DBA expertise will be more valued than ever.
In the near term, the need to manage the wave of increasingly critical machine functions will be the overriding factor. In the long term, the software will take a while to catch up with the hardware (as usual), so DBA skills should be in high demand for at least another five years.
The Internet will also profoundly impact the DBA job description. DBAs will interact with peers and seek advice through e-mail and online conferencing. Vendor support will center on the World Wide Web. Patches, updaters, and entire software products will be sold via the Net.
In the last 10 years, our industry has become a place where change is successfully promoted as "good" by those who can profit from it. The synergistic combination of the vendors' need to sell, the press's need to publicize, analysts' need to proclaim, consultants' need to get clients, and IS managers' need to show "immediate cost savings" will continue to drive the industry to one new technology after another, even before the technologies prove their worth at real-world IS sites. In other words, we'll continue to grab one silver technology bullet after another whether or not it fits the gun.
To DBAs, this means a serious time commitment to learn new technologies. DBAs will also need the insight to judge which trends will take hold quickly (client/server database), catch on over time (object-oriented programming), or fade into niche oblivion (AI). Continual self-education will be the new mantra of the successful DBA.
Undergirding technological change is the sociological change that has occurred in the workplace over the past 10 years. Whether downsizing and the absence of corporate loyalty to workers will continue is anyone's guess, but one thing's for sure: Now that the genie is out of the bottle, it won't go back in again. Managers will continue their short tenures, employees will lack the assurance of continued employment, and the ranks of contractors will continue to swell. DBAs will have to keep their skills current, broad, and deep; smart DBAs will leave positions that fail to promise professional growth. The strongest resumes will include many projects at many companies, not longevity in a single position at a single company. Last but not least, as the democratization of the workforce continues, male DBAs will stop wearing suits and ties to work, and female DBAs will burn those "male mannequin" suits from the '80s.
The fragmentation of skill sets accompanying new technologies is a related trend. Ten years ago, a DBA could learn a single DBMS-DB2, for example-and have a great career. But today's heterogeneous environment demands knowledge of multiple databases. Furthermore, most DBAs must also be familiar with multiple communications protocols and operating systems. Single-skill DBAs face the future at their peril.
As the skills crunch becomes more severe, wise IS managers will develop new hiring criteria. They won't look for specific skills as much as broad experience and the ability to adapt quickly to the product du jour.
Short-term managers with short-term horizons will continue to mandate projects that can be completed only in short time frames. (With an average tenure of less than three years, few CIOs will plan longer projects.) More DBAs than ever will work in deadline-driven, unplanned environments. Massive internal application-development projects will be the exception rather than the rule, and DBA understaffing will be chronic.
Although this business culture will be frustrating and difficult, it will also give DBAs their greatest chance to shine. Pandemonium means freedom.
Overall, the future will be intense, exciting, frustrating, confusing, and profitable. And did I mention fun?
Howard Fosdick is a DBA, systems administrator, and independent consultant.
You can reach him via e-mail at [email protected].Back to top
Yuval Lirov and April Langer
Most recent DBPD feature: "Database Support: Can It Be Measured?" July 1995
Database technology is evolving toward reduced support costs and improved performance. Users and application developers will be the ultimate beneficiaries of the new technology, which will achieve levels of standardization and personalization that empower Internet users to create and maintain their own shared database resources for home or office.
According to Moore's Law, the capacity of a computer chip doubles every 18 months. Assuming this law holds for the next 10 years (as it has for the previous 20), database searches or indexing that take 12 hours today will take less than four minutes in 2007. Cheap holographic memory will hold terabytes of data in less than a cubic centimeter of space. And as Bill Gates predicts, a single wire will deliver mountains of digital data to every home.
Such progress will profoundly change database technology in terms of data content, database administration, and the DBA profession. Data content will evolve from records of text and numbers to temporal sequences of images, voice, and text (movies, that is). Data access methods will also evolve beyond traditional tables of data toward concepts such as the Informix DataBlades, enabling direct manipulation of complex data objects. Finally, many traditional DBA tasks requiring skill and experience today-such as replication and data integrity verification-will be automated.
Database administration is a relatively new concept. In the late '70s, client/server technology introduced the idea of the systems administrator, which combines the knowledge of systems internals with the access permissions of the systems operator. The DBA concept originated with that of the shared database in the '80s; it combines knowledge about application requirements with the ability to arrange the data so as to optimize response time.
Because database availability and many performance issues are independent of the logical schema and can be treated in a general way, the complexity of the client/server environment split DBA responsibilities into logical and physical components based on the principle of proximity to application code. While the "logical DBA" continues to model data and understands the application code, the "physical DBA" spends most of the time improving database availability and response time regardless of the logical database schema. (Hence the natural affinity between systems administrators and physical DBAs.)
To satisfy growing demand for shared computing resources, improved applications availability, and support scalability, DBAs will use a two-pronged approach: distribution and centralization. First, distributed data exchange mechanisms will replicate databases in the client cache, eliminating query network traffic and parallelizing queries. Next, centralized navigation tools will direct applications to the primary servers, expediting emergency failovers without application code changes. Integration will be the tactic for implementing this strategy; DBAs will use a combination of technologies from different vendors. To survive in this new environment, vendors will have to demonstrate specialization as well as interoperability in their products.
The DBA profession will become more general to facilitate teamwork with systems administrators and developers. At the same time, it will become more specialized to meet the growing needs of increasingly sophisticated database technology (OLTP, OLAP, data mining, and warehousing). In a way, every developer and user will perform some basic subset of today's specialized DBA activities. Consider the example of telephony: In the beginning of the century, the demand for phone operators grew at a rate that would have required every American to be working as an operator by 1950. Instead, the invention of automatic switching alleviated the pressure while transferring some of the operator duties to users. A similar process will occur for DBAs.
However, internal systems support continues to suffer from a resource schism: Its benefit is transparent to the users while being a necessary evil. Because such support is not a core business for the enterprise, users always perceive its costs as exorbitant for the value of delivered systems. Continual budget cuts coupled with growth in systems-service demand result in insufficient support-resource allocation, which reduces systems-service quality and client satisfaction.
As this demand for support services grows, the enterprise will lose the ability to keep pace with the need for in-house support services. Rigorous support-quality and productivity metrics will receive widespread acceptance. The support profession will evolve from art into engineering discipline-becoming more competitive, gaining professional prestige, and earning better compensation. (Salaries for support professionals are already growing faster than in any other computing services area.) Furthermore, the increased need for specialized services and for growing quantities of data will result in even more sophisticated and complex databases. In fact, products are already so complicated that vendors need to secure appropriate knowledge base in the market prior to sales.
Some vendors have opted for internal professional service units to meet the systems-integration demand. However, such services suffer from the same schism as the internal support services: Because client support and systems integration are not the core vendor business, resource allocation seems too high for the product developers and yet too low for the service providers. Such services can be effective and efficient only with adequate cross-platform and cross-vendor tools, with integration with other support disciplines such as systems administration, and after spending significant amounts of time at user premises.
In contrast, economies of scale for database integration are possible through a three-pronged strategy: consolidation of user basis, specialization of products to meet special market needs, and integration of products to work with multiple vendors. Spin-off of existing professional services divisions seems to be the most likely option.
In summary, database technology will continue to evolve in the context of sweeping changes in the information industry and support culture. First, the invention of new and improved products will foster specialization and competition. Next, the growing complexity of disparate platforms and vendors will fuel the development of effective and efficient integration methodologies. Finally, efficient systems support will enable specialization and increased competitiveness, reinforcing the synergy among computing industry, support culture, and database technology.
Yuval Lirov is a vice president of distributed systems support at Lehman Brothers in New York. He is author of the book Mission-Critical Systems Management (Prentice Hall, 1997). You can reach him via e-mail at [email protected].
April Langer is a senior vice president of global technology support
at Lehman Brothers. You can contact her via e-mail at [email protected].
Back to top
Tim Quinlan
Most recent DBPD feature: "Report from the Trenches," December 1996
When I look back over the last decade, it seems the DBA's job hasn't changed that much: Although DBMS technology may be vastly different from that of 1987, the role of the DBA is not. I do, however, think this role will change significantly in the next 10 years-but not in any revolutionary way.
First of all, we'll be spending more time doing the things that we should be doing. More emphasis will be placed on physical database modeling and testing for performance, capacity, and configurations, as well as on performance simulation and stress testing. The greater emphasis on these areas will improve the quality of our systems and enable us to spend less time on operational issues.
Second, the future DBA will work less overtime. (I can almost hear the cries of disbelief out there.) Sure, there'll be some; but working day and night as an ongoing habit will be a thing of the past. I know that we pride ourselves on playing "catch-up" in 60-hour work weeks, but let's get over it and take pride in doing a great job in a 35-hour week.
The DBA in 10 years will also spend less time programming and writing scripts, because tools will be available to speed up this work. Indeed, the increased use of database procedural code such as stored procedures and DataBlades doesn't necessarily mean that DBAs will write more code. (I personally don't believe in the concept of the "procedural DBA.") This work can be performed by application developers, with the database standards being set by and database calls reviewed by the DBA.
The placement of application code and business rules will continue to be an issue: Should they be stored in the middle tier of an architecture or in the database itself? The introduction of application servers, Web servers, DataBlades, Data Cartridges, relational extenders, and new development tools cloud the issue further. Most application development will be performed with tools that enable modeling, code generation, and application partitioning in which most of the code is put in the middle tier. DBAs will have to understand these tools and take a key role in determining where application logic should be placed. We will also need to work with object-oriented and new (currently unknown) modeling techniques to do our job.
There will, however, be a great deal of code implemented in databases. Most of it will be purchased from third parties and will deliver specialized functionality (fingerprint analysis, for example) designed for a specific database platform.
Furthermore, databases will give the illusion of becoming a commodity item in several respects. For example, operational support will become more streamlined and provide better price-performance ratios: We'll be supporting more and larger systems with higher availability requirements but with fewer people. In many cases, operational support will be performed offsite by outsourced resources that monitor the system remotely.
Databases may also appear to be a commodity in the area of smaller systems, on which almost any database and design will perform-at least in the short term. Fortunately, we'll develop a greater respect for the problems associated with small systems that have been quickly slapped together, as some of the smaller client/server systems developed in the 1990s were. Overall, we'll have a greater appreciation of the benefits of system architecture and design.
Nevertheless, anyone who tries to convince you that databases are commodity items will have forgotten the many other issues that make managing them even more complex. Indeed, the contrary will be true: We'll be delivering larger databases and supporting more users with very high availability, security, and performance requirements. And we'll have to deliver at an ever-increasing pace.
The DBA job description will become more specialized as the gulf between "warehouse DBAs" and "enterprise/OLTP DBAs" widens. This dichotomy already exists but will become more pronounced as the technology evolves.
We will also have more options to choose from when designing for multimedia data types. Diversity will flourish; the huge amounts of data being pushed through our systems by an increasing number of users will force us to use different modeling techniques, scale-up options, DBMS types, physical configurations, optimization, and indexing to accomplish our goals. DBMSs will continue to have subsets and supersets of standards, so the challenge of managing heterogeneous environments will continue to grow. Improved middleware and management tools will ease some-but not all-of this pain. For these reasons, the DBMS will not become a commodity item, as some will claim.
In short, databases will become even more complicated than they are now, and DBAs will become more specialized. But our work will be even more interesting.
Tim Quinlan is a database consultant in the Toronto area. You can
contact him via e-mail at [email protected].
Back to top
Prepare to convey database requirements to people with
incredibly short attention spans
Casey Young
Most recent DBPD feature: "Seeking the Promised Land," June 1995
Assuming we get past the Year 2000, I foresee that DBAs will have multiple roles to fill. The good news is that we'll be able to choose the type of DBA we want to be. The bad news is that these roles will proliferate, and regardless of how many DBAs a company has, the work will still have to get done. Unfortunately, those of you who feel splintered now and have too much to do will be under even greater stress-unless you can convince management that you need help.
In 10 years, members of the "Nintendo generation" will be well entrenched in our business. More prone to rapid development and decidedly PC-oriented, they'll look for DBAs who are quick to react and able to view an architecture from multiple angles. These new DBA jobs will call for more "people" skills than some of the other, more traditional roles. If you have one of these jobs, you'll have to maintain an impressive balancing act. You'll need to be highly aware of changing technology (beyond what you read in the trade press) while effectively communicating traditional database requirements (availability, performance, and data integrity) to people with incredibly short attention spans looking for the latest gimmick and technological "gotcha."
Part of the act, however, is to appreciate the qualities this new generation will bring to the table. Sometimes "quick and dirty" works; sometimes long-term objectives can only be served by tried-and-true methods. To succeed, you'll have to know the difference.
Working hand-in-hand with these business-oriented DBAs will be the implementation DBAs. These DBAs will need to understand the technology to the tiniest detail. If this role appeals to you because you want to avoid communicating with people and stick to machines, think again: No DBA in the 21st century will get by without communication skills, because there is already more technology than one person can memorize. Twenty-first century databases will not just sit on the mainframe monolith and hum; rather, they'll be integral to the complex technology feeding information to stations around the world. Communicating with other reluctant communicators will be critical when problems arise.
A third type of DBA will also emerge before the end of this century. A byproduct of the emerging object/relational paradigm, this DBA will be the "gatekeeper" who ensures that access to the database falls within rigorous performance standards. (If you're an SQL power user, you may see your job evolve into this type of DBA function.) With the advent of stored procedures, triggers, encapsulation, and so on, I foresee a proliferation of small objects masquerading as "reusable code." When companies realize that everyone has a different idea of which object to use for a particular problem, order will be restored and the power-SQL DBA will be on top again.
Where the systems DBA is concerned, the importance of network and LAN connections is sure to grow. Already DBAs are connecting databases to the Internet and to help determine the communication protocols that enable tolerable access time. As new data types requiring additional communication behavior (such as "streaming" for video) emerge, understanding how it all fits together will be critical.
In addition, to develop the best communication protocol for database performance, you may be required to troubleshoot complex systems. Locating a performance problem when the only complaint is that too much time elapses between the time the user presses "enter" and the time an answer is returned is a very frustrating proposition. In the new world, someone will have to be able see the whole picture-from client workstation to network protocols to database activity.
Your mission (should you choose to accept it) is to do some self-analysis and determine where you fit in this new world. Keeping up with inevitable change is a big responsibility, but the days of doing "the same old thing" will soon be over.