So last post, I gave an introduction to virtualization and left off with a small teaser saying that I would discuss my preferred method of deployment. I call this architecture 3DV which stands for 3PAR, Dell and VMWare. I'm fairly certain that anyone reading this article is familiar with Dell and is aware that Dell provides a full line of business hardware. VMWare is somewhat less ubiquitous than Dell, but they provide what I believe to be the best virtualization software to date in the form of their VI3 (Virtual Infrastructure 3) line. Least known of the three is probably 3PAR which provides an excellent line of innovative storage area network (SAN) hardware and software. The combination of these three provides a solid high-performance virtual infrastructure which is flexible on various levels.
This post will cover 3PAR. The backbone of any large infrastructure is storage. Storage needs often grow faster than computing power needs, and sharing storage is key in today's tightly integrated IT environments. There are many SAN vendors out there like EMC and NetApp, but 3PAR differs from these in two key areas - thin provisioning and dynamic optimization. By nature, most SANs force you to allocate all storage space needed by a server at the time you provision it. This means that even though your server may only use 100 GB currently, if you expect the need to grow to 500 GB at any point in the future, you need to allocate 500 GB immediately. Naturally, this leaves about 400 GB of storage space allocated but not used. This is a waste of that space. Multiply this schema across several to many servers, and you can see that a lot of space gets wasted. This practice of allocating more storage than currently necessary is called over provisioning. 3PAR addresses this issue with a technology called thin provisioning. Conceptually, the 3PAR unit presents a logical unit (LUN) to the server of any size you specify, but only fills up the space that is necessary at that point in time on the actual SAN. This means that the server may see a storage space of 1 TB, but the 3PAR is only utilizing 100 GB. The other 900 GB that would otherwise have been immediately allocated to a server and removed from the available storage of the SAN are not actually removed from the available storage pool. They are still available to be used as necessary by the SAN. With this type of provisioning, system administrators can 'thin provision' storage space out to servers up to the physical limit of the SAN, yet not actually have purchased and installed all of the disks that make up that limit in the SAN itself. The system administrator must monitor the actual usage on the SAN and purchase and install more disks as needed. Thin provisioning greatly reduces unused but already purchased and installed storage space on a SAN allowing IT departments to squeeze the most bang for the buck out of their SAN infrastructure. Dynamic optimization conceptually spreads all data on the SAN over as many disks as possible to increase the throughput of data from multiple disks (spindles) and thus increase IOP performance. Thin provisioning and dynamic optimization give the best performance and least cost in a SAN infrastructure.
It goes without saying that 3PAR provides the usual array of SAN goodies like robust connectivity options including iSCSI and Fiber Channel. 3PAR also provides a command line interface (CLI) and an easy to use web-based graphical user interface (GUI.) There are multiple units available for medium to large enterprises and many software tools to enhance the capabilities of the SAN. You can find more information about 3PAR at http://www.3par.com
Virtualization - 3DV the 3PAR SAN
Virtualization - Streamlining the Data Center 3DV 3PAR Dell VMWare
I'll start off this series by pointing out something I find funny every time I write an article on virtualization. Most word processors will mark the word 'virtualization' as misspelled. There is a good reason for this. When the majority of standard dictionaries in use today were developed, there was no such word as virtualization. How do you virtualize something? How do you take a physical object and make it virtual? Is this something like virtual reality? Actually, it is.
In the world of IT, virtualization means taking a physical server and turning into a virtual one. The concept of virtualization is that these 'virtual servers' run on an intermediary software layer which in turn runs on top of the physical server hardware. That intermediate layer then presents virtual server components to be used by server operating systems that will reside on top of the intermediate layer. These virtual components include processors, RAM, NICs, CD-ROM drives,hard drives, USB devices, video cards, sound cards, etc. In the virtualization world, the physical server with the intermediary software layer on it is referred to as the 'host,' while any operating systems installed on top of that virtual layer are called 'guests.'
Virtualization allows IT departments to consolidate several servers onto a single piece of hardware. This is especially useful for small application servers, print servers, web servers and any other server that is not extremely resource and input/output intensive. I must point out, however, that such resource and I/O intensive servers can be virtualized with the right (read expensive) hardware. The true benefit of virtualization comes as old and new servers are provisioned in this virtual environment, and the total number of physical servers needed in the data center is reduced. This also has a positive environmental impact by reducing the amount of hardware, electricity and cooling necessary to keep a data center running. As you can see, virtualization has the potential to reduce costs in many places for IT departments while increasing the efficiency of data center hardware. Virtualization also provides some very nice disaster recovery and high availability options because virtual machines can move between physical machines. There is a lot more to virtualization that this quick introduction will not cover, but be rest assured that I will discuss virtualization in much more detail over the next series of posts. One of the great benefits of being a consultant is implementing virtual infrastructures for companies and watching their reaction as they realize the full potential of a virtual infrastructure. As a teaser for what's to come, I architect all of my clients' virtual infrastructures using 3DV infrastructure. 3DV stands for 3PAR, Dell and VMWare. You can look into each of these companies and get a sneak peek into what I will be covering in this virtualization series. I will also explain why I use 3DV and ways to implement a virtual infrastructure in your organization if you do not have 3PAR or Dell hardware. Until then, good luck!
Sphere: Related ContentEvery Student in the Republic of Macedonia to Use Ubuntu-Powered Computer Workstations
Wow! It never ceases to amaze me where Ubuntu pops up in the world. Here in the United States we have a huge bias toward (or against in some circles) Microsoft. Outside of the United States, however, open-source software is taking off at phenomenal speeds, and Ubuntu seems to be leading the way as the distribution of choice.
Popular Linux Distribution To Appear On More than 180,000 Classroom Desktops Nationwide
LONDON, November 20th, 2007 - The Republic of Macedonia Ministry of Education and Science will deploy more than 180,000 workstations running Canonical’s Edubuntu 7.04 as part of its "Computer for Every Child" project.
The Republic of Macedonia "Computer for Every Child" project is one of the largest known thin client and desktop Linux deployments ever undertaken. Half of elementary and secondary Republic of Macedonia students attend school in the morning, and half attend in the afternoon, so 180,000 workstations will allow for one classroom computing device per student for the entire Republic's public school population. The first 7000 computers pre-installed with Ubuntu were shipped on September 4th 2007
Ubuntu is the award-winning Linux distribution for the desktop, laptop, thin client and server which brings together the best of open source software every six months. It is free, community-developed and contains all of the necessary applications to be up and running immediately, from web browser to instant messaging. It also comes with English and Macedonian language packs.
"The Computer for Every Child initiative is the largest and most important education project undertaken in the 15-year history of the Republic of Macedonia," said Ivo Ivanovski, Republic of Macedonia's Minister for the Information Society. "By selecting Ubuntu as the operating system for all of our classroom virtual PCs, our education system can provide computer-based education for all school children within the limited financial and infrastructural confines that most institutions face today."
"The Republic of Macedonia is showing tremendous vision by deciding its own destiny in choosing how its children will learn through computing – said Mark Shuttleworth, founder of the Ubuntu project. "By choosing Ubuntu which is free to modify, use and redistribute it turns students into participants in Information Technology and not simply consumers of it.
The Ubuntu operating system will run on 160,000 virtual PC terminals and 20,000 PCs (which each also support a student on the attached monitor) supplied by NComputing and procured and installed by The Haier Company, a diversified manufacturer and PC maker based in China.
The project will enable a range of innovative educational programs, including interactive web-based classes in which specialized experts teach lessons in such areas as mathematics, biology, chemistry and physics to multiple schools and classrooms around the country.
Sphere: Related ContentThe Model View Controller (MVC) Architecture
Web Architecture is always a fun area to dabble in. I have had the fortune of being part of this exciting fast-evolving environment for upwards of a decade. What I love about being a systems architect in the web environment is that although new languages and frameworks enter all the time, the tried and true elements of web infrastructure have remained fairly constant. I still work almost entirely within LAMP stacks or Microsoft stacks. The mainstay of the web is still HTML (regardless of how it is rendered.) Scripting languages and corresponding frameworks are still powering the dynamic web as we know it, whether it is Ruby on Rails or Cake PHP. Mashups are all the rage these days, and with that paradigm of blending old and new, the MVC architecture returns to the forefront of web development.
The Model View Controller, or MVC for short, is a software architecture pattern which solves an age old problem of separating data access and business logic, the data itself and the User Interface (UI.) In the MVC, the UI is the View, the data itself is the Model, and the intermediate component that separates business logic and data access from the View and Model is called the Controller. This architecture pattern is not new, however. It was first introduced in 1979 by Trygve Reenskaug while working on Smalltalk at Xerox labs. His original work was titled Applications Programming in Smalltalk-80(TM):How to use Model-View-Controller. It seems that today the rage in building new large scale websites and web-based applications has centered itself around the MVC architecture.
We can define the MVC as follows:
- Model -The domain-specific representation of the information that the application operates. Domain logic adds meaning to raw data.
- Many applications use a persistent storage mechanism (such as a database) to store data. MVC does not specifically mention the data access layer because it is understood to be underneath or encapsulated by the Model.
- View - Renders the model into a form suitable for interaction, typically a UI element. Multiple views can exist for a single model for different purposes.
- Controller - Processes and responds to events, typically user actions, and may invoke changes on the model.
Taking our definition of the MVC components from above, we add some more detail to the components:- Model - The model represents enterprise data and the business rules that govern access to and updates of this data. Often the model serves as a software approximation to a real-world process, so simple real-world modeling techniques apply when defining the model.
- View -The view renders the contents of a model. It accesses enterprise data through the model and specifies how that data should be presented. It is the view's responsibility to maintain consistency in its presentation when the model changes. This can be achieved by using a push model, where the view registers itself with the model for change notifications, or a pull model, where the view is responsible for calling the model when it needs to retrieve the most current data.
- Controller - The controller translates interactions with the view into actions to be performed by the model. In a stand-alone GUI client, user interactions could be button clicks or menu selections, whereas in a Web application, they appear as GET and POST HTTP requests. The actions performed by the model include activating business processes or changing the state of the model. Based on the user interactions and the outcome of the model actions, the controller responds by selecting an appropriate view.
As you can see, the MVC architecture is very powerful and truly scalable. When put into the context of large web sites and web-based applications, it really shines. Many different scripting languages and frameworks are now taking advantage of MVC architecture. Two of my favorite are Ruby on Rails and Cake PHP. I hope this introduction to MVC architecture helps people who are currently involved (or will be involved) in large projects that will build applications on top of the MVC architecture. As always, I am happy to discuss my posts in more detail via email. Sphere: Related Content
Blackberry Enterprise Server and Microsoft Exchange Interaction
I was recently introduced to a problem involving a Blackberry Enterprise Server (BES) and Microsoft Exchange Server 2003 in which the Exchange Server was acting up and the source of the problems was semi-obscured. The symptoms of the problem emerged on the Blackberry Server with worker threads failing health checks. After about 5 or 6 failed health checks, the Blackberry Service would restart and the issue would repeat. At firs the Blackberry device users were not receiving emails in a timely manner. At a certain point, they were not receiving emails at all.
I dove in head first to see what was going on. From the fact that that worker threads were failing health checks I came to the conclusion that the issue was on the Exchange Server side. Each thread issues Remote Procedure Calls to the Exchange Server and waits for these RPCs to terminate. The RPCs were timing out and as a result the health checks were failing.
On the Exchange Server, the RPC traffic was very high, with spikes maxing the server out. It was very evident that the Exchange Server was inundated with RPC requests. The irony of the matter is that the Blackberry Server was mainly responsible for those RPC requests. It is a common assumption that Blackberry Server traffic causes upward of five times the RPC traffic of a normal Outlook MAPI client. You can see where this can cause a vicious cycle.
My next step was to find out why the RPCs were not terminating in a timely manner. I looked at the RPC latency and that was high as well. I eventually narrowed down the problem to disk contention issues at the storage array on which the Exchange Server mail store databases were housed. The array attached to the Exchange Server was due for replacement soon anyway, so I decided to move all of the logs and databases to a super-fast SAN that was recently purchased for these types of servers. Suffice it to say that after the move to SAN, all of the issues were resolved. Because of the blistering speed at which Exchange could read from and write to the SAN, the RPCs were terminating in a timely fashion. I saw latency drop to single digit values from triple digit values. The Blackberry Server was no longer having worker threads fail health checks. Although the RPC traffic remained the same, the lower RPC latency improved overall performance by an order of magnitude.
I would like to note two things for Exchange or Blackberry administrators to be aware of here. The first is the very obvious point that a high-performance SAN greatly increases performance and reduces problems over all. The second is that when talking with Blackberry Enterprise support, the tech repeatedly tried to convince me that the Blackberry Enterprise Server did not have that great of an effect on an Exchange Server. He flatly denied the 5 fold increase rule, and he even said that it was a myth. He did hit the nail on the head when he suggested that the performance issue was on the Exchange Server side. I want to note, however, that when I turned off the Blackberry Enterprise Server to rule it out as a cause of the issue, the RPC requests dropped through the floor. The moral of the story is that although there were multiple reasons that contributed to the overall problems, never take the word of vendor support at face value. Always test every possible cause of the issue in a methodical fashion. You will probably find the root cause faster and end up much happier as an end result.
Three Tier Web Application Framework
One of the topics I often get asked about by clients and server administrators is the three tier web application framework. Although it is pretty straightforward to describe, there are some important considerations when deploying a three tier web application framework, especially if it is meant to scale to a very large implementation. The trend today seems to be moving toward a model control view (MVC) framework, but that is a totally different framework that I will cover in another post. Essentially, the three tier web application framework seeks to separate the web application into three parts, which run on three different physical machines, and utilize the power of the three pieces of hardware to speed up the process. An additional but crucial benefit of this model is that it separates the process into tiers that can scale individually to handle large amounts of traffic. Let's start out with a picture to illustrate the three tier web application framework. This picture comes courtesy of tqnyc.org.
As you can see from this illustration, the three tiers that make up the three tier web application framework are the client tier, application tier and data tier. The interaction between these tiers is well defined, strictly enforced and rarely circumvented.
The client tier is probably the most diverse in terms of possible configurations, but it is pretty much the same with respect to structure across its variations. Each client has its own operating system. A web browser runs on top of the operating system. The Sun Java Runtime Environment is often running on these systems as well in order to extend functionality beyond simple HTML rendering. Some additional pieces of software may also be installed such as Flash to enhance the users' multimedia experience. Typical operating systems at this tier include various distributions of Linux such as Ubuntu, RedHat and SUSE, Microsoft Windows and Mac. Typical web browsers include Mozilla Firefox, Safari, Opera and Internet Explorer.
The application tier can also be of various composition, but the typical setup is Linux with Apache and PHP. This typical setup is often referred to as a LAMP stack (minus the M which is part of the data tier.) A similar implementation with Microsoft Windows Server, Internet Information Services (IIS) and the .Net framework is also widely used. Various other platforms exist that will run on the Linux/Apache and Windows/IIS stacks such as Ruby on Rails. All of these frameworks are meant to extend usability and provide for rapid development.
The data tier is comprised of the database or XML files that hold all of the data for the system. Databases are the most widely used applications in the data tier, so we will focus on those. In a LAMP stack, the M stands for mySQL which is an open source database application. On the Microsoft side, SQL Server (current version is 2005) is the database application of choice. There are other alternatives such as PostgreSQL which focus on specific database functionality needs. The SQL, or structured query language, is what allows the application tier to search the database and read or write to it. The database remains the single repository of all data in the system instead of having a plethora of random files stored all over the place.
As you can see, the three tier web application framework is not too difficult to dissect and understand. When referring to the LAMP stack, often an administrator will run all of the stack on a single server. This is fine for a small web application or web site that will not have heavy traffic. If you plan to have large amounts of traffic such as a social networking site, the separation of each tier is what allows you to scale the framework. By placing all data on a database, that database can later be cloned and mirrored many times over. This results in multiple databases with identical data being served to a multitude of web servers. These large groupings of databases and web servers are often referred to as farms. Just as one farm may grow a lot of corn and another may grow a lot of wheat, these server farms may have many databases or many web servers in each. In a large scale environment, hardware load balancers such as F5's BigIP take incoming requests from clients on the web and spread them evenly among the web servers in the farm. This is a truly large scale three tier web application framework. This type of distributed application provides stability, speed and redundancy - all of which are vitally important to a business who is deploying this type of framework. There are various other ways to tweak and implement the three tier web application framework, but they are often more complex and outside of the scope of this post. This post is meant to provide information for someone who is not current with the web frameworks available. If you need more information, I am always up to discuss web frameworks via email.
