I work for a large multination company, which boast possibly the largest intranet in europe.

The microsoft parts of this intranet are taking, upto 9,000,000 hits per day, using an assumtion of 10 hits per page impressiion (GIFs, CSS etc) that is 900,000 page impressions per day, I can’t speak for the linux hosted parts as I don’t have access to ALL of the data.

Compare to  an avarage joe-doe blog site or even the avarage small business web site (4,000 page impressions per day, this is real data from a small 10 employee ecommerce web site) that is quite impressive.

Then enter the BBC , from the BBC annual report “bbc.co.uk’s reach grew by 16.2 per cent over the year, with page impressions averaging over 3.6 billion per month.” that is 120 Million page impressions per day (or using the same assumption I made above 1.2 billion server hits per day)

one word

WOW !!!!!!!!!!!!!!!!

Due to the fact it is on my task list, I have been pondering the decision points for deploying a new web based application and supporting infrastructure. At a first stab there must be 400 questions which may be in the mix. The trick is to work out which ones are Critical to success, which ones have no impact, which ones can be linked and therefore become a single decision with many results. Does the implementation mean some of these decisions can be made after Go live (VMware lets you add extra resource to a guest operating system)
How much input can the business customer provide?
How much input can the application developer provider?
What assumptions can you make?

How much dynamic processing is going to take place, (this requires MORE CPU than static)?
How many page impressions per hour are going to be required?
How many hits per page are required?
Do they need to host large Files, podcasts, video etc?
Do they need to RUN reports on the Data?
How many concurrent sessions are going to be supported (this may impact memory)?
Is the developer using any .NET memory intensive data manipulation?
Do they require Web HA or a WEB farm?
Do they require SSL?
How do you deal with SSL in Web HA or a Web Farm?
How do you deal with Session state in Web HA or Web Farm?
Will they require HA or web farm in the future, (your design must scale out)?
Is the application data read or data write intensive?
Is the Application Developer trusted to write safe code?
Can you lower the .net trust settings to protect the Os from the application?
Are file uploads required (this can impact scale out)?
Local or NAS based content (NAS content will require extra identity configuration)?
Where do you keep the web server logs?
How long do you keep web server logs?
What level of logging do you use for the Web server?
How much pro-active monitoring do you do?
How much Local disk space do you need?
What extra .NET assemblies are going to be required?
What extra COM components are going to be required?
Does the developer understand he is not going to get console access?
Does the App need scheduled tasks?
Does the APP use DTS and / or SSIS?
Does the App need a large TempDB?
How big is TempDB?
How big are the database logs files?
Does the App need to partition the tables?
DAS or SAN for the data disks?
IF SAN, one big LUN or lots of LUNs (does it make a difference)?
Does the app need database mirroring or Failover cluster?
IF Database mirroring is done, is it HA or high perf?
If database mirroring do you use a witness for auto failover?
Does the app take a data feed from someother system?
is this a linked server?
is this a SSIS job?
is this a regular bulk import?
Are there any BLOB object being used?
Can you trust the App developer to setup indexe on appropriate columns and with correct fill factors?
I will add more when I think about them !

In todays must be green society many corporates are looking at ISP style web hosting farms to consolidate the Intranet  , extranets and Internet web sites onto fewer servers with better engery cost and perfromance.

Building a large web platform(s) is not something you should take lightly, it is a not a simple case of “right click BOSH”, you have to concidure many things

• How things may interact?
• How you control it?
• How you track who owns what ?
• How do you charge for it?

So I have started to compile a Web farm checklist, which will grow over the next few months as i try to document some parts of the last 4 years of my working life.

 General points to ponder,

• CMDB, what do i need to record
• What sort of load balancer (on web server (NLB) , hardware (Cisco ACE ) or software (ZXTM))
  Session affinity (stickiness) (clientIP, cookie insert, ASP session cookie or none and use some other session management approach)
• IIS site instance numbers to be unique across every IIS Web platform so you can move a site from server to server or platform to platform without confusing logs file folders
• Use of CIFS / DFS for back end content store, so you can upgrade and move FILE servers without have to change any web server configs or apps code, no need to replicate large amounts of DATA between local storage
• Centralised FTP service , provided single FTP server for apps owners to access any site on any platform, use of DFS simplifies management here too, IIS6 FTP required phantom folder to make the virtual paths browsable, FTP7 can fix this
• Use of domain Account for worker process (this is a requirement if using CIFS (UNC SHARE) based content
• NO application write access except to “writablefolder” or “Private” which may of may not be outside of the web browsable file space
• Log files, if using LOGfile to generate stats how long to you keep them for, and where
• Logs file to include ALL Data, this is a must if trying to analyse application performance
• Do you offer detailed log file analysis , number and frequency of 500, 404 etc, this is very easy with logparser and some simple scripts
• Standard set of components supported, only provide one component for EACH business function (Create PDF, Send Email, Upload something, Base64 encode)
• Do you need SSL, if so do you terminate at the web server or at the load balancer (I prefer the latter)
• Do you provide a SQL database, MSACCESS does not work well in a load balanced environment
• Do you offer PHP ? (PHP now has a MS MSQL client)
• Do you Cold Fusion ?, if so how, Adobe Cold fusion on J2EE or Blue dragon CFM on .NET
• .NET Security config, how much to you pin .NET down
• File Space Quota (will depend on CIFS solution)
• SMTP how to you manage this, give it away for free, or a small charge per message
• Automation to create NEW, amend and delete sites
• Standard memory offering for worker process, how to you monitor and report any worker process recycling
• Do you provide CIFS Snapshot / Previous Versions
• Do you provide a Development instance, as standard, to stop live testing !
• If so does the developer control the code release to production, or do you have a “deployment” system with change management
• Installation of .net service packs and patches, how do you test with over 1000 apps on a platform
• Use of centralised config server, keep ALL of your IIS config  on one offline server  and deploy setting via XML to live servers, it is possible to use automation and a DB to drive most of the values needs to create sites, but in a corporate environment someone will always want something changed outside of the automation design
• Do You need to promote asp.net session service and move away from stickiness
• How do you manage DNS Name space

For IIS7

• Do You need to look at automation for single level app folders as this is the only thing you don’t seem to be able to delegate
• Do you need a set of guideline on IIS7 pipe line integration
• Do You need to look at not allowing certain IIS7 pipeline integration?

Possible Gothca’s

• Use of CIFS back end file store on IIS6 required increase NetBIOS command stack values see http://support.microsoft.com/kb/810886/
• Use of individual User ID for Each site will cause desktop heap issues see http://support.microsoft.com/kb/831135
• If you are using a centralised config server, all file paths HAVE to be the same, c:\windows\system32\inetserv for example, you can use Junction point to get around this, if you don’t control the OS build, but you need to be careful around path in web extensions (metabase value 2168) as they MUST be the same as the paths in the script maps (metabase value 6014)

Not only do you have to concidure the technology, There is the non techie stuff too

• Standard Look and feel for ALL sites
• Search engines (a nightmare for an intranet)
• Document management
• Content Management
• Content Review
• Content Quality
• Content Accessibilty

There are some Great non techie points about intranets here

Notes to self

• Write up and publish FTP design for IIS6 (and IIS7)