Google - Don't be evil is bullshit

What should have been one of the Internets greatest success stories, Google has in more recent times come under scrutiny about its business practices and ethics.  A company that in 2004, set out with the inspiring goal to make sure that all of Humanity's information could be accessed from its Search engine.  Now finds itself coming under criticism as it broadens its reach beyond Search and Advertising into Telecoms, Mobility and Social Networks to name but a few.   


A company that aspired to a value system that became well known as "Don't be evil" has been replaced by a new motto asked by many now -- "Is Google evil?".


 Back in 2005, i actually believed that Google was a benevolent company, and blogged about it here (http://captaink.blogspot.com/2005/09/talk-about-google-talk.html).   


I have changed my position of that over the last few years.


It hapenned with Microsoft, in the late 90's, when they grew too fast, became big, became arrogant and now, ironically, act as a good internet citizen with humility.  There seems to be increasing evidence that Google is operating like the early Microsoft years.  Now you might say what makes Google evil?


Several examples to cite:


1. Patent-Theft:  Eric Schmidt then the CEO of Google, sat on Apples board between 2006 and 2009.  The very time when Apple had been preparing its iPhone into the market in 2007.  It turns out that Google had previously purchased Android in 2005 (before the iPhone came to market) as a mobile operating system to compete directly with front-facing QWERTY physical keyboard devices like Blackberry and Windows Mobile phones.  Android was designed from the outset for keyboard devices, which did not have a full-screen, or multi-touch gestures.  Google later stole many iPhone multi-touch features and included them in Android.  Steve Jobs in his last days promised to "go thermonuclear on Google... to right this wrong".


2. Disingenuous:  When Vic Gundotra, said: 

"If Google didn't act, it faced a draconian future where one man, one phone, one carrier, were our choice.  Thats a future we don't want....  If you believe in openness, if you believe in choice, if you believe in innovation from everyone, then welcome to Android".

We are led to believe that consumers have a choice between a curated, tightly managed AppStore where content doesn't contain malware, tracking cookies, porn vs. the Google free 'open' Android marketplace.  Thats the real choice with Open -- it doesn't seem so attractive now does it?  When Gundotra says that Android is open, it means as a handset maker you have access to latest source code?  that as a handset maker you can contribute back into the main source code?  err no.  In fact Google says that you must ask permission and explain to Google exactly what your plans are before receiving the latest source code.  How is that open?  When Gundotra says that "it turns out that on the internet, people use flash" - what he didn't want to admit was that Flash sucks on mobile phones and are drain on processing power and battery life.  How ironic today, 2-years later, with the just released Chrome browser on Android, they chose not to support Flash.


Being disingenuous -- doesn't make Google evil, but it portrays a company that will exploit a populist viewpoint to characterise itself to be benevolent, to give the impression that its the vanguard of technological idealism. 




3. Monopoly:  In 2008, DOJ filed a file suit against Google at its unlawful attempts to tie search advertisement prices with Yahoo.  In 2010, a federal district court in new york agreed with the DOJ again at Googles attempts to monopolise book content by forcing publishers to give Google exclusive access to content.  Google has imposed technical constraints to make it hard for Microsoft Bing search engine from correctly accessing sites like YouTube - in effect Google is creating a walled garden on prime internet real-estate.  Lets take where Google is very dominant and that is in search advertisements - they make it almost impossible for advertisers to export their ad campaign data to another advertisers.  Over Googles second failed attempt at social media, the FTC alleged that Google used "deceptive tactics and violated its own privacy promises to consumers" over the use of consumer data with Google Buzz. 


4. Patent War:   Google is being chased by Apple, Microsoft and Oracle for patent infringement in Android.  Apple view is simple - multitouch is a competitive advantage that they will protect aggressively -- and they have chosen to attack Google by indirectly suing handset makers like Samsung and HTC.  In what has become a patents arms race, things escalated when Google purchased Motorola with the sole aim of acquiring its patent chest, a desperate attempt after it had failed to win the Nortel patents.  None of these Motorola patents could seriously be used to defend Android against Apples claims.  So why spend $12Bn on a company losing $250M each quarter?  It turns out Google is willing to hold the industry at random by allowing Motorola to to request unreasonable license fees (2.5% of selling fee of products) on FRAND licenses (FRAND licenses are meant to be reasonable license fees to enable telecommunications interoperability in the industry).  

Feb 2012 - So the Google acquisition of Motorola was approved, but take note of the carefully worded statements from:

(a) the US Justice Department - "the company has not made sufficient promises to license essential patents in fair, reasonable, and non-discriminatory (FRAND) terms".
(b) EU Competition Commissioner - ""This merger decision should not and will not mean that we are not concerned by the possibility that, once Google is the owner of this portfolio, Google can abuse these patents, linking some patents with its Android devices. This is our worry. … We might be obliged to open some cases in the future. This is not enough to block the merger but we will be vigilant."

5. Social Media Search:  Right now if you search the Internet using Google -- you get a biased internet search result that will favour Google properties.  Don't believe me try it out.  Try and find and content from Twitter -- where is it?  Same crap as we hear before, Twitter claims that Google doesn't need permission to index Google.  Wrong according to Google -- Twitter needs to provide certain metadata before Google can index Twitter.  Well this seems like a special rule that Google makes for special competitors like Twitter.


Should i stop here or carry on?  Let see how the rest of 2012 plays out shall we....


In the meanwhile recent postings (10 ways in which google runs the world, The case against Google, Google gets big for its britches, Google: The Hero becomes the Villain, Is Google facing the beginning of the end?, all seems to coincide with many things of the things i have been saying here.   

Time to move on....

first Robert Scoble, then Bill Gates, and now Om Malik.... all within the space of a week....

is there some kind of cartel or collusion hapenning, where everyone agrees to ditch in their job at the same time for maximum news exposure?

I guess Gateas has realised that curing Aids in Africa is more important than the next operating system. But, why-oh-why, did he not make that noble decision about 5-years ago? -- it would have been better for everyones sake!

Ray Ozzie memo: Microsofts Services strategy

Dave Winer has posted 2 e-mail memos, from Bill Gates and Ray Ozzie, on the next step forward for the company.

Clearly Ray has had time to talk to the product teams, senior management and developers and has finally taken stock of the situation.

Ray Ozzie mentions some key tenents:

1. The power of the advertising-supported economic model.
2. The effectiveness of a new delivery and adoption model.
3. The demand for compelling, integrated user experiences that “just work”.

Fundamentally, Ray's memo is a more than just a call to developers and product teams about service-enabling their assets. Its a an action-plan. Driven by end to end user-driven scenarios, something that Ray is very good at, from these use-cases, he will drive architectural design decisions and map that to current and fututure products/services.

I think this is not a good time for Microsoft, yet you can't but feel that finally they have someone at the top of Microsoft that actually gets it. What Mcirosoft has delivered as Ray has said is a 80% solution. In the post-"Vista/Office12" wave we will see a fantastic level of seamless integration of technology that weve not seen before, his past efforts with Notes and Groove should give us some inclination of the potential.

I have still some doubts though; Microsoft needs to balance Enterprise revenue models with Consumer/advertising revenue models. There needs to be harnomy between product and services -- the old mindset to lock-in services and products just won't cut it.

Windows "Live"

Fantastic! -- Reported by TechCrunch. Its Windows even when its not Windows! "Windows Live (beta)" is Microsofts go-to-market branding that will encompass its next geneartion SAAS (Software As a Service) strategy.

Its the harmonisation between Microsoft Windows, Microsoft Office applications, and Microsoft Services (Windows Live). We have seen this brand used in "Live" Meeting and XBOX "Live".

As Ray Ozzie mentioned here, the iPod and the Blackberry were "perfect examples" of how to create a tightly designed user experience from hardware, to software to services.

I think we will see a great example of that with the XBOX 360, a huge investment by Microsoft which goes beyond just gaming, it could easily become the centre-point for many of Microsofts other strategies around building tightly coupled hardware/software/services.

In terms of the announcements yesterday?
- early 90's - Microsofts energy was put into creating Windows "New Technology" (NT)
- mid 90's - Micorsofts energy was put into embedding Internet/Media experience into Windows.
- 2000's - Microsofts energy was put into .NET and Web Services strategy.

They stumbled a bit after that, and had this grand vision of connection people, devices, and services through "hailstorm".

The last 5 -years has been painful partly through trying to converge technologies streams into a big bang product roadmap, most of which will be released soon, .NET 2.0, Visual Studio 2005, Vista, SQL 2005.

But in the next 5-years, we'll see Microsoft tinkering less in that space, with putting more of a concerted effort on the User experience (not User Interface / not Product!) but my Identity, my Devices, my Work, my Perrsonalisation and my Events.

We will see a new abstraction of "the Windows API". It won't be Win32 or WinFX. But more likely WinLive Web Services API. All XML based contracts.

Ray Ozzie spent a long time talking about Ad Revenues being a $150bn industry by 2015. So wheres the lock-in? That bit is not clear. You'll still need Windows to truly use these Services. Microsoft needs to tread carefully here, but i suspect they are realising that lock-in through their software platform (to generate license revenues) isn't going to run in the long-term (not outside the Enterprise anyway), its going to be about converting users into revenue generating consumers for advertisers.

MSN used to be a place where Microsoft used to place executives before retirement. If i were to believe how Microsoft is positioning MSN against Windows Live, its clear to me one is designed for syndicated content (where current advertisers are at) and the other eventually will become the place for contextualised content (and where advertisers might go to in the future.)

Y! Messenger & MSN Messenger

So... When the large Portal players (who in the past never gave a damn about interoperability) all of a sudden decide to play fair and interoperate. -- What are we supposed to think....?


Sunnyvale, CA, and Redmond, WA -- October 12, 2005 -- Yahoo! Inc. (NASDAQ:YHOO - News) and Microsoft Corp. (NASDAQ:MSFT - News) today announced a landmark agreement to connect users of their consumer instant messaging (IM) services on a global basis. The industry’s first interoperability agreement between two distinct leading global consumer IM providers will give MSN® Messenger and Yahoo!® Messenger users the ability to interact with each other, forming what is expected to be the largest consumer IM community in the world, estimated to be more than 275 million strong.

That all of a sudden they give a damn about us? err no.
- More likely that Microsoft couldn't get the right deal with AOL.
- More likely that MSN and Yahoo! look at this pragmatically as a way keeping their respective registered users, rather than lose them.

As mentioned at Om Maliks Blog,
"Somehow with a move like that, and a potential AOL/MSN alliance it is starting to look like we will be having a three way dance between the Portal IM players [excluding AOL], Skype and Google."

Federated Identity Management

Well ok. Not the most interesting subject.... But nevertheless here i was, sitting on the train on my way to work... and guess what... im listening to these folks from IBM, Microsoft, Sun and HP talking about Federated ID Management... What the heck!

I have not payed much attention in this space for some time... well not since the days of Microsoft Passport and Sun's Liberty Alliance (if it wasn't an alliance then.. it sure is now).

Anyhow... I got thinking that i'd actually forgotten what it was like to listen to people talk. You know that kind of chatty... comfy... night time radio chat ... Ovaltine moment... Right now most forms of communications rely on me actually writing or reading something... we use IM, SMS Messaging, e-mails, blogs... so it was pretty refreshing to hear these folks talk on my iPod.

I got to hear these legends who are responsible for others things including the WS-* specs talk about:

• SAML, WS-Security, Liberty, Oasis.
  
• Microsoft and Sun's differing view on how standards should be created.
  
• Convergence between Liberty and WS-Security.
  
• Why ID management is important enough that there aren't differing standards.
  
• I liked Microsofts idea of a more informal trust management solution (Infocard) that meets the needs of ad-hoc/social/personal/collaborative situations, much like what is used in Groove and PGP) - Although the Liberty folks thought that Enterprises weren't interested in that... (i think thats an old skool security mindset.)

Anyhow getting back to Ovaltine... I like idea of an online directory based - offline radio broadcasting data cache. This time round the revolution will be Podcast.... right?

So Don

I noticed this little snippet from Don Boxs spoutlet...

Earlier this week, I wanted to install a significant piece of Microsoft software on my machine. The first step of the setup program was not "Install" but rather "Plan." When I selected that step, I was prompted by a ~100K HTML file with all of the prereqs, QFEs and side effects I needed to understand before installing. Given that I have almost no additional time nor room in my brain to process this data, I opted to not install the software and instead dedicate that time to letting the product manager know why "he lost me at hello." Hopefully it will get fixed.

Is it me or is he getting just a little bit irritated with the pre-reqs for installing BizTalk 2004?

Of course this is all just speculation.... :-)

How to learn about BizTalk 2004

People have been dropping me e-mails asking me what is the best way to get into BizTalk and how to learn it.... My advice is to stop reading blogs, and start playing with the code! Only kidding!

To that end the i would highly recommend downloading the newly updated BizTalk 2004 SDK. There are loads of examples that include (orchestrations convering compensation, adapter patterns including the SQL adapter, using the BAM API, invoking the Business Rules engine, Custom Pipelines, building custom Functoids, creating WMI scripts). Its a fantastic resource, so get your head buried in that stuff and you'll be fine...


Also to reduce anyones day-to-day stress, its worth keeping an eye on the Microsooft Knowledge Base for known issues with BizTalk 2004.

Also MSDN has posted some new articles on process integration using BizTalk 2004 and service-oriented integration

More deliberations around Bus topology with BizTalk

Scoott Woodgate has posted an excellent whitepaper that describes the guts of BizTalk 2004, particularly around the architecture of the MessageBox and Logical Hosts.

The question i posed to Scott, was that within the context of subscriptions, how does this 'BizTalk Group' or 'MessageBox network' work across an enterprise WAN? My earlier understanding came down to how practical will it be run a globally distributed MessageBox architecture, are there latency issues that make this unworkable, and therefore we need look other multi-cast approaches to delivering an ESB?

From a BizTalk perspective does that mean we have no choice, but to employ Zonal BizTalk Groups, and all the implications that go with that from an Operational/Admin/Routing point of view.

thoughts please...

XML Web Services in BizTalk 2004

Joe Klug from Microsoft, had lots of new snippets on the Web Services support in BizTalk 2004 server:

1. Publishing a Web Service
Through the BizTalk Web Services Publishing Wizard.
(i) expose an orchestration or a schema as a Web Service.
(ii) support of unknown SOAP headers.

2. Consuming a Web Service
Add Web Reference
(i) through this you can re-use and aggregate existing Web Services.

3. SOAP Headers
- SOAP Headers are supported for published and consumed web services. A known SOAP header is as defined in the WSDL, and unknown headers are sent in message but not defined in the WSDL (so BizTalk wouldn't know about them), and aren't supported when consuming a web service.

4. WSDL extension
- WSDL Extension - supports 2 SOAP extensions. You can replace the schema defined within the WSDL; this is useful since due to a limitation in the current version of ASP.NET (will be fixed in Whidbey) the generated file can produce some loss of information during the class conversion (e.g. maxOccurs and minOccurs).

5. Type-Agnostic Ports
- BizTalk 2004 allows for the creation of Type-Agnostic ports which is a port that can receive any type of XML document:-
(i) Its means that the Message type is an XML Document (System.XML.XMLDocument .NET namespace)
(ii) Such a document could also be a multi-part message type.

Here is a pattern of how you could abstract the web services layer across all your BizTalk subscriptions.

(i) You could create a dummy orchestration using this type-agnostic port which receives and sends a XML document.
(ii) You could publish it as a web service, but never actually deploy that orchestration.
(iii) You could create multiple orchestrations that subscribe to different types of message, but only have single web service receive point for multiple different message types.

6. WSE Adapter for BizTalk 2004
- Hopefully by the end of 2Q2004 (Beta in Feb/March) Microsoft will release a WSE Adapter for BizTalk 2004. This will be based on WSE 2.0, and be fully configured through policy assertions rather than code. WSE standards for Web Services Extensions, and provides a mechanism to deliver a product-grade Web Services stack that can be used across server Operating Systems and products. Since OS release-cycles occur every 3-4 years, WSE contains the latest Web Services specifications. Therefore this BizTalk adapter will piggyback on the WSE life-cycle, and will very likely become the future 'Indigo' adapter for BizTalk. There will need be some form of migration from WSE to the Indigo Message bus (can't say Indigo!)

7. SSO confusion in Sharepoint and BizTalk 2004
- Microsoft BizTalk 2004 Server and SharePoint both employ different solutions for Single-Sign On (SSO), from what is understood SharePoint will change its SSO solution to support BizTalk. It will be possible to generate BTS SSO tickets when using Web Services (SOAP headers).

Microsoft are in the process of writing a whitepaper to clarify this all, since architecturally its a bit confusing that there are different approaches to SSO, including an agreement between the two product groups going forward about supporting a 'Single' single sign-on solution (well done Microsoft - duh!)

BizTalk 2004 and Microsoft position on ESB and Indigo

Below is some discussion about Microsofts definition on what is or isn't an ESB, and how that relates to BizTalk 2004. Included are some comments from Don Box on Indigo, and how Indigo fits in with the SOA model. Scott Woodgate then pitches in on the BizTalk strategy to leverage the underlying platform. And how this will continue in the future with the support of Indigo adapters for cross enterprise connectivity.

This thread appeared (although its not there now) on microsofts public forums microsoft.public.windows.developer.winfx.indigo and first reported by winfx247.com

Question/
Will this still be a function of BizTalk and if so how will you guys reposition BizTalk from EAI to ESB. As I understand it Indigo will serve as a message bus, I think it will be important for you guys to soon make it clear what you plans are in this area.

Answer/
Don Box [MSFT] (VIP)
I'm not sure what ESB is, but here's the Indigo/BTS story.

Indigo can work with existing BTS implementations in that they both support SOAP and WSDL.

The version of BTS after Jupiter will use Indigo directly. This brings Indigo's security, RM, and TX stack to BTS.

DB

----- Marlon Smith wrote: -----

Will this still be a function of BizTalk and if so how will you guys reposition BizTalk from EAI to ESB. As I understand it Indigo will serve as a message bus, I think it will be important for you guys to soon make it clear what you plans are in this area.

Christian Weyer
I guess by ESB he means Enterprise Service Bus ... just like adding a new
word to the SOA buzzword bingo :-)
http://www.cbdiforum.com/public/news/index.php3?id=1326

Cheers,
--
Christian Weyer
[Microsoft Regional Director, Germany]
[MVP ASP.NET & XML Web Services]

** XML Web Services: http://www.xmlwebservices.cc/
** Weblog: http://weblogs.asp.net/cweyer/

"Don Box [MSFT]" wrote in message
news:52D46B5F-B004-4BE6-BB3F-63168434F7F5@microsoft.com...
[Original message clipped]

a message bus, I think it will be important for you guys to soon make it
clear what you plans are in this area.

--
Christian Weyer
[Microsoft Regional Director, Germany]
[MVP ASP.NET & XML Web Services]

** XML Web Services: http://www.xmlwebservices.cc/
** Weblog: http://weblogs.asp.net/cweyer/
--
Scott Woodgate \(MS\)
--
I'll jump into this thread as the BizTalk guy. What Don says is correct but
I'd like to break it down for everyone:

- Indigo provides support for secure, reliable, transacted "web" services
which is obviously pretty important :)

- Indigo unifies ASP.NET, DCOM, COM+ etc. This point is important because
today BizTalk Server 2004 uses ASP.NET ASMX APIs under the covers for its
web services support (we didn't build our own WS stack we leveraged the one
in the platform and provided value add infrastructure around it for
transformations, graphical point and click etc.) and tommorrow we will
transition those APIs to Indigo

- Indigo is a transport in the terminology of BizTalk Server (just as MSMQ,
MQSeries, Web Services)

- Because Indigo is a transport and BizTalk Server has an extensible adapter
framework for transports (application or technology) I demonstrated a
prototype at PDC an "Indigo" adapter (cf. the Web Service adapter we use
today). Adapter is probably a funny word to use its just the generic point
in the architecture to plugin transport support the support will be native
in the next generation in the "Longhorn" timeframe when Indigo ships.

- Adding to that Indigo is an API (not a set of tools). One of the cool
things BTS2004 provides even in an all "web services" environment is super
rich tracking infrastructure and a variety of design/management tools.

- BTS2004 will continue to support heterogeneous "messaging" <- which is an
overloaded word in case you didn't notice microsoft message queuing does
messaging biztalk server does "messaging" they are different but related
things, same thing with indigo which provides the migration path from msmq
and biztalk server, and of course orchestration/workflow moving further
forward (and a bunch of other innovations :))

- BizTalk Server will continue to support EAI, I'll dodge the ESB point
because there are various analyst interpretations there, and SOA as it
does in 2004 in the future obviously enhanced with the underlying platform
that Indigo provides.

My PDC presentation for those who are interested is at
http://blogs.gotdotnet.com/scottwoo. I'll probably put up a video of the
demo in the next few days. The important thing here is that both BizTalk
Server and Indigo are innovating together to move forward the platform and
the server in the "Longhorn" timeframe and your investment in BizTalk Server
2004 is a strong investment to move forward.

Cheers,
Scott.


--
Scott Woodgate \(MS\) (VIP)
--
From a BizTalk Server perspective we are all about leveraging what is in the
platform and then adding value on top of it. As these lower level management
features/functionalities become available in the platform we will leverage
(and actually in some cases enhance them directly) and continue to innovate
on top of them.

Cheers,
Scott.

BizTalk and Publish/Subscribe, EAI and the Enterprise Service Bus concept

So every-1 is talking about enterprise service bus architectures (basically pub/subscribe) and how that is a more cost effective and easier approach to EAI than the traditional hub/spoke model. Well first off, the notion of an ESB was first banded around by Sonic Software (of SonicMQ fame), and then legitimised by Gartner and more recently by IBM. Sonic have a product called Sonic ESB that they believe differentiates them from the traditional EAI vendors.

Their definition of an ESB as i could understand it is that it must have the following characteristics:-
- must be a 1st generation product (i.e. built after 2000)
- it must be built from the ground-up using XML, XSD, XSLT standards.
- MOM(asynch, pub/sub, store & forward) / Transformation / Content based routing.
- Highly connected (e.g. Web Services / J2EE)
- Monitoring (Helath and Activity Tracking)
- Distributed deployment.
- Single point of management / control.
- Location / Technology transparency.
- Fault Avoidance (ease-of-use) / Fault Tolerance (intelligent routing) / Redundancy.

In simple terms here is how ive differentiated ESB and EAI.
ESB - Simple intelligent routing, distributed peer-based architecture.
EAI - MOM, explicit routing, central hub, service broker.

On the face of things it looks like BizTalk 2004 has a lot of these features, but im not yet sure that BizTalk has a fully distributed deployment architecture that provides location transparency and a single point of management, i may be wrong - ( im just writing my current thoughts!), but i think the issue may related to how you can distribute BizTalk servers across an enterprise with multiple data-centers (over a WAN).

BizTalk 2004 has the concept of a BizTalk group (a domain or federated group in other words), which is effectively the scope for Administration, Routing, Message Tracking and all other services. All related BizTalk receive and processing servers are part of this scope. All servers within a particulary BizTalk group use the same MessageBox. This is an important point, since how and where you deploy your servers are impacted on the fact of the single MessageBox design.

Lets think about this, once a message is received (through the Receive Port and Receive Pipeline) it ends up in the MessageBox (where any meta-data like promoted properties of the schema are stored). Any interested parties ( whether its a map, rules engine, an orchestration or even a send port/pipeline) can subscribe to that message.

The MessageBox relies completely on SQL sever for its persistence, and since BizTalk does have a scale-out story by having multiple MessageBoxes (you still need to have a master MessageBox database - i am planning to test this.) you may be able to deploy those MessageBoxes in a distributed manner, although im not sure that is its primary design. I assume that it was designed to improve the scalability of the primary Master MessageBox. In any case there are better ways of improving the scalability, namely scale-up SQL Server (more on this later).

So BizTalk allows you to have multiple MessagesBoxes, you may be able to distribute these across the enterprise (although this may not be its primary design, and may introduce considerable latency issues), but assuming it does, it means that all the distributed BizTalk receive/processing servers can be part of a single BizTalk Group, and if that is the case you get the complete benefits of location transparency, and a single point of management.

BizTalk 2004 is starting to heat up...!

Just received the Release Candidate on BizTalk 2004 (BTSEvalRC - 74367-000-0000007-05479). It sure has been tidied up. Not only that, but there seems to be generally more chatter on the forums and with a lot more material popping up.

Anyway this entry will be a placeholder of references (heres a few for starters):-

• BizTalk PowerToys from luke nyswonger.
  
• El Grego ( i don't know his name) has a sample which shows how to use the Enterprise Instrument Framework to generate an application event (in this case a suspended message caused by a send port retry failure). Although BizTalk generates a plethora of WMI events that can be picked up by MOM (and NetIQ soon).
  
• Martijn Hoogendoorn has a Suspended Queue Listener Windows Service which basically listens out for the WMI event and allows you to then do what you want with it... It works in real-time and batch mode as well... Get it here.
  
• Brian Loesgens has some cool code that implements service windows for send/receive locations. Get it here.
  
• Stephan Bouillon has a real cool component that replaces the standard XML validator used in the BizTalk default pipeline. Rather than the standard behaviour of suspending the message it sends an error to the messagebox, this allows you to have a error handling orchestration that can subsribe to these messages... This greatly improves the end-to-end management of BizTalk, product team why did you not think about this!!?!
  
• Another bright chap Stephen Thomas has some interesting pieces on sequential convoying, etc...
  
• Jeff Lynch is doing some pretty harcore mapping examples, especially around conditional outputs for linenumbers.
  
• Giles Zunino has a good example of how you can archive any messages as they are passing through the BizTalk pipeline. This i see as a standard service that ought to be provided by BizTalk, and perhaps through HAT you can achieve this. However companies may have compliance issues which deem the need for a more legally watertight solution. Anyhow something that i had been thinking about also.
  
• The Core Engineering team from the Biztalk product team are now getting on the case, this really does raise the quality of information coming out. No other vendor can match this openness. Well done Kevin B Smith and Lou Graber
  
• Jan Thielens Blog has some good best practices on BizTalk 2004, particularly around foundation components like error logging, exceptionstions, and XML dis-assembly (since the product documentation only covers CSV)
• Mike Holdorf's been keeping it real with some pains and best practices on BizTalk.
  
• Charles Young from Solidsoft has some interesting articles on BizTalk Orchestrations and Transactions.
  
• Carlo Poli has is having some fun with versioning BizTalk assemblies over here...
  
• A new book on BizTalk 2004 development on Amazon.
  
• Another Christoph has some development learnings on using BizTalk orchestrations.
  
• The Middleware company has released a .NET community for serverside, enterprise specialists, its called strangely enough, theServerside.NET!!
  
• Some early ideas on naming standards for BizTalk orchestrations.
  
• Scott Woodgate who is the programme manager of the BizTalk Server / Jupiter Programme.
  
• An interesting presentation from Claesen Christof on BizTalk 2004 and how it can be used as a Enterprise Service Bus. And another on a simple and easy BizTalk 2004 overview.
  
• Darren Jefford is having some fun with MSMQT and MSI Exception handling at his blog over here.. way to go!
  
• The gotdotnet community has a number of code samples and its own special workspace for BizTalk.
  
• Mike Woods has his own msn workspace which include the hands on labs that was available at TechEd and PDC.
  
• You can find many of the presentations and videos from the PDC 2003 here . (its only here for 6 months so hurry!)
  
• You can find many of the presentations and videos from the various TechEds 2003 here.
  
• Benjamin Mitchell has some interesting things to say regarding Indigo, BizTalk and Yukon (Service broker). More on his comments later.
  
• Microsoft's BizTalk 2004 beta newsgroup.
  
• Microsofts BizTalk public newgroup on all things BizTalk.
  
  More to come soon...

WinFS - Why its a big deal..

The following snippet from Ray Ozzie succinctly sums up why he reckongs WinFS is a big deal. These things take time, but WinFS will open up some really fascinating changes that we can expect in the next 5 years (or so). The 'network-effect' of WinFS, assuming we end up ubiquitously storing primary schemas types about at least our identities and profiles, means we will be able to do some pretty amazing things with personal communication and collaboration. Searching my digital assets, (i.e music, photos, documents) against standard schemas that make cross-referencing, searching and inferencing of my content, and content that i am interested in, extremely easy. IMHO, this makes WinFS the most significant part of Longhorn.

"No - the "big deal" about WinFS IMHO isn't Search. Like Web Services - it's about the fact that it's an attempt to get a higher level of interoperability between programs through agreement on schemas. Hopefully, toward the goal of bootstrapping network effects, and unintended/innovative consequences, on the client. WinFS defines an extensible object model and persistence mechanism, as well a rich and extensible "relationship" mechanism that can be used to intertwingle objects with that are somehow related to one another. Some kinds of stock relationships are obvious: common Author, common Artist, common Location, common Priority, etc. Some may be more subtle or domain-specific: common Project, common Client, common Contributors, or even manual and thus not-easily-described ties." More....

Tech.Ed03, Barcelona, July 2003 - presentations

Phew...

Just finished posting my ramblings from my recent visit to TechEd 2003. My postings are in the order in which i viewed the presentations, so as you go further forward, you may see some duplication on the material that was presented.

All in all. I enjoyed this years Tech Ed presentations. So what was i able to take back about Microsoft and their products:.

• Windows 2003 just works. They have built on top of the stability of the previous release to offer (i) NLB and improved clustering, (ii) .NET framework 1.1, (iii) Web Services & UDDI, Virtual Shadow Copy, ....and much more im sure!



• Office 2003. I didn't attend any of the seesions on Office, but from what ive read to date. I am impressed. I think the way that they are opening up Word using XML and XSD will really expand the way people will start to build producitivty applications. That coupled with InfoPane, which provides a context sensitive client side panel embedded in Word, which allows developers to build workflow based applications that leverage on XML and Web Services. Additionally they released InfoPath, an XML Forms editor that can be used send XML data to BizTalk... I really don't know where Office 2003 and InfoPath are going... but i think it will really open up new oportunites for building apps... i would recommend taking the time to follow these new developments.



• BizTalk 2004. Of course, you could not go to TechEd without seeing the announcements for BizTalk Server 2004 (which by the way is an absolute darling of a product). This is more a reflection of how how quickly Micrsoft has taken leadership position in the EAI space in a very short space of time. While companies like SAP are making noises of their NetWeaver / SAP XI platform strategy, and IBM still in the process of integrating (and rebranding to IBM WBI) its recent acquisition of CrossWorlds and its ICS Server / Collaborations product... Oh and lets not talk about TIBCO's continual acquisition of products (last one being Talarian?), it sort of makes me think that these companies are really struggling to deliver a cohesive strategy. So I think Microsoft have really stolen a lead, with its organic development, that is in line with its product stack and their Jupiter product strategy. This is additionally supported by Microsoft entering the Gartner Magic Leader/Ability to Execute Quadrant, which means they must now be taken very seriously!

BizTalk 2004 introduction

Design goals
- Focus on making integration projects easier.
- Provide standardized way of building the solution.
- Provide standardized ways of making information available to information workers.
- Accelerating Web Services as way of unlocking legacy systems.
- BizTalk 2004 is Project Voyager (Jupiter v1) – Jupiter roadmap converges the other Windows Servers systems (e.g. Content Management, and Commerce).
Standards
- In BizTalk 2000/2002 documents were supported using XDR, BizTalk 2004 now supports XSD .

- The Web Services-Interoperability group (WS-I) gathers a set of standards known as profiles that can work together in an interoperable way (e.g. SOAP 1.1, WSDL x.y, HTTP 1.1, WS-Security, etc.). In BizTalk 2004 the WS-I basic profile is supported.

- BizTalk 2004 has a number of standard adapters automatically built into it (e.g. FTP. SOAP, SMTP, etc), and allows for newer adapters to be easily updated into the product.

- All schema mappings are executed internally using XSLT.

- BizTalk 2004 orchestrations can be imported and exported using the Web Services orchestration standards BPEL .

- BPEL is also GXA compliant

XML web services
- Compared to BizTalk 2002, which had 7 different development tools, all development in BizTalk 2004 uses Visual Studio 2003.

- Consequently, all the BizTalk ‘components’ are compiled to IL, and managed as a standard a .NET assembly. (Well mostly)

- Additionally a BizTalk orchestration can itself be exposed as a Web Service using ASP.NET

Architecture
- Additional adapters are expected at the time of shipping BizTalk 2004.
- BizTalk 2004 has the concept of a Message Pipeline (they used to be called Channel’s in BizTalk 2002) that provides the decode/encode, decrypt/encrypt, parse, and extract of the messages.

- After a message goes through a Message Pipeline it ends up in the Message Box (in the past with BizTalk 2002, such state was stored in many places like the database, filesystem, WebDAV). This is all now persisted in MessageBox.

- In summary a typical solution flow would like: -
TransportàReceive pipelineàMessageBoxàOrchestrationàSend pipelineàTransport

- Built on top of MessageBox are additional services like Correlation & Content Routing, and Activation and Instance Management.

- Tracking tools and monitoring tools have been substantially improved from Biztalk 2002.

- Information workers have additional services that they use BizTalk for, namely around Business Intelligence (business activity monitoring, and human workflow)

MessageBox
- The MessageBox is central to the BizTalk architecture.
- All messages are persisted on SQL Server 2000 SP3.
- The messages are treated as rich objects with properties.
- MessageBox is based on stateless pub/sub architecture.
- This makes it very easy to scale out.
- Correlations are a way of declaratively identifying messages through its complete lifecycle within BizTalk (i.e. how do we know that this PO is related to this Acknowledgement)
- Large message support (can stream messages up to 2Gb without any problems.)

Scalability
- System is mostly stateless. (Most of the routing occurs inside SQL Server)

- Except for the following:-
o Persisted orchestrations, (long running transactions)
o Message correlations, (must be routed to correct machine – in some cases)
o Hydration/dehydration,
o MSMQ (session based protocols)

- Can use NLB (windows or hardware) to scale front-end to the MessageBox (SQL servers)

- MessageBox is therefore the bottleneck; however they can also be scaled out. Although one of the MessageBox must be the master MessageBox.

- The Master database MessageBox can therefore take advantage of standard SQL Server clustering; this is an improvement over what could be done with BizTalk 2002, which was quite difficult.

Security
- MessageBox is secured, stops other hosts from accessing it.

- Individual BizTalk processes can be targeted to specific hosts (secured using Service Accounts)

- No guest messages to be allows through the receive pipeline (must be authenticated - through this default behaviour can be disabled).

Developer user experience
- Visual studio is de-facto user experience for anyone working with BizTalk 2004. (Mapper, Editor, Orchestration designer, Pipeline designer)

- Schema mapper provides 98% of what is normally done with XSLT. (Also possible to launch to an external XSLT engine.)

- Functoids based on .NET (drag and drop)

- The BizTalk orchestration designer is much more sophisticated in BizTalk 2004. BizTalk 2002 had 6 shapes whereas BizTalk 2004 has 22 shapes.

- BizTalk orchestrations can now be composed of other orchestrations, making the task of designing the orchestrations more manageable compared to the cumbersome and complex approach in BizTalk 2002.

- Parallel actions. Looping constructs. Compensation constructs (long running.)

Information worker
- Exposed functionality through Ms-Office 2003, specifically using InfoPath 2003, it will be possible to route XML documents using Web Services through a BizTalk orchestration.

- Business Activity Monitoring (BAM) are a set of services that rely on SQL OLAP to analyse the data from the documents and/or processes to provide the business information worker with a view on the business process in real-time.

- The BAM information is presented to the end-user using Microsoft Office tools.

- BizTalk 2004 also contains a highly scaleable business rules inference engine.

- An Information worker can be more proactive about how the business is running, my taking the metrics that the business uses, and then run through the orchestration, analyse them using Business Activity monitoring, and then re-apply changes to the business rules based on the newer information.

- The rules composer is used to create the rules. The rules are always version managed, so any changes are made to newer versions, and not the deployed ruleset.

- The deployment of the ruleset is very easy, and independent of the BizTalk deployment assembly. This allows the business users to modify the ruleset without having to re-deploy the BizTalk orchestrations.

- The separation of the business rules from the business process makes good sense, since the business rules change more frequently than the business process.

Trading Partner Management
- BizTalk offers an improved SEED package, which is an easily distributable BizTalk package for trading partners (includes SharePoint Team Services, and InfoPath).

- BizTalk makes it easier to manage relationships with thousands of trading partners and separates this from the business process.

- Additionally, BizTalk 2004 introduces the concept of Service Links. Service Links offer a way dynamically binding a port to a partner at run-time, based on partner or message data. This is different to content based routing where the end ports are already bound.

Human based workflow
- Human-based-Workflow is a new feature provided in BizTalk 2004, which provide a layer of services on top of the Business Activity Monitoring Services in BizTalk 2004 that allow Office 2003 to track human based Office workflow activities.

- These human based workflows are characterised by the fact that they are task oriented using tools like Office 2003; that the workflows can be modified in run-time; process is ad-hoc and semi-structured;

- Human-Based-Workflows leverages the BizTalk Orchestration engine, and built on top of BPEL.

Automating Business Processes and Web Services using Microsoft BizTalk Server 2004

Orchestration is document-centric using XSD rather than XDR. Concurrency, decision branching, transaction processing are additional components available in the orchestration.

New in BizTalk 2004
- Integration with the .NET framework.

- Orchestration designer generates MSIL. (Fully compiled unlike BizTalk 2002, which was sort of interpreted.)

- Orchestration and Messaging engines are now combined.

- Deployment model is based on the standard .NET assemblies using Visual Studio.Net

- More complex business processes can now be constructed.

- Can compose orchestrations (abstraction to other orchestrations.) – allowing developers to build a library of orchestrations.

- Custom and advanced correlation.

- n:m schema mapping (now you can map several schemas to several schemas n:m)

- Flexible binding options (early, late, dynamic)
2002 just had early binding- static addressed for testing.
2004 now has late binding - for orchestration binding.
Note: (This is separate to port binding, which can be done later closer to time of physical production.)

- Move from 6 different tools in BizTalk 2002, down to a 1 single tightly integrated tool.

Basic features
- Send & Receive Message - building blocks for the business processes.

- Ports - named locations where we will be sending and receiving messages from (addresses, URI)

- Conditional Branching based on Boolean (decision) or message type. (Listen)

- Document Transformation (can integrate with the BizTalk mapper)

- Parallel Actions (fork and join, concurrency / synchronization)

- While Statement (while xyz do this… etc. etc. etc.!)



Advanced
- Transactions
ACID transactions (same as what was in BizTalk 2002)
Long Running, compensation.

- Exception Handling (in BizTalk 2002 wasn't very rich model, now improved vastly, more programming centric C# model)

- Service Links (a concept in BPEL4S - a collection of ports that describe a particular action with your trading partner. At runtime you can determine trading partner to communicate with rather than coding port at design time.)

- Flexible Business Policies. (This refers to the rules inference engine capability.)

Port Binding
In BizTalk 2004 there are a number of options for binding ports:

- Specify now (URI, filename, etc.)
- Specify later (at some later time, the administrator determines where this message will be received from)
- Direct binding (communicate with other running orchestrations instances, or talk to the pub/sub MessageBox) … not clear on this one –SK.
- Dynamic (the reply to address is determined at runtime.)

- Using the BizTalk Orchestration designer a developer can easily drag a web port type and link it to a Visual Studio code Web Reference. In effect using the BizTalk designer a developer is able to bind a web port to a WSDL interface, which is bound to the auto-generated proxy code generated by Visual Studio.

Web services in BizTalk.
- Web Services Publishing Wizard – allows you to publish an Orchestration as a web service. By selecting the port that you want to expose, and by defining a target namespace, SOAP headers and Single-Sign on services. Visual Studio will create an ASP.net project.

- BizTalk 2004 provides features that allow for a more robust environment for building web services, for example when an Orchestration is exposed as a Web Service, BizTalk will track and ensure retries to make sure of guaranteed execution.

Administering, Deploying, and Managing BizTalk 2004

BizTalk administrator
- MessageBox’s are assigned to a SQL Server database.

- Can now isolate BizTalk Orchestration Applications, MessageBox, Processing, Adapters, and Tracking onto specific hosts.

- These Hosts can then be mapped onto server instances.

- ReceiveLocations - where document flow in. (E.g. SharePoint, file, etc.)

- SendPortGroup - multiple subscribers (1:M)

Hosts - default host’s setup when BizTalk installed. "BizTalkServerApplication"

Note: The whole area regarding how BizTalk can scale up/across was brushed over rather quickly, and therefore its unclear how BizTalk 2004 takes advantage of any of the new features of NLB cluster and Server cluster capabilities in Windows 2003 or whether its independent of BizTalk.

Deploying Applications
- Developer use Visual Studio to produce Orchestrations, Pipelines, Schemas, Maps, and Rules. By using abstractions of the physical locations, and then wrapping them up in .NET assemblies.

- Scaling issues can be more easily managed when the administrator connects them up to the real physical world.

- BizTalk explorer is used to create the physical environment to the logical BizTalk application. (i.e. orchestration ports to physical ports, and mapping of applications to Hosts)

- Deployment wizard tool (is not part of Visual Studio) is used to create a deployment script. The tool ensures that the BizTalk assembly is registered in the GAC .

Demo
Use Visual Studio to deploy application à puts the assembly into the GAC. à Then use BizTalk explorer to set-up the send and receive ports. à Specify the physical locations. à
After that you need to bind the orchestration to the newly created send/receive ports.

How to move from dev to test to production? à Use "BizTalk Deployment Wizard" application à "export .NET assembly binding to a file." à specify assembly, specify location to place the BizTalk assembly binding file. à Creates xml script. à How to deploy to multiple machines? à Add a new project within visual studio. "BTSInstaller" à Build the project to create an MSI package.


Securing BizTalk
- BizTalk relies fully on SQL Server roles for access permissions (e.g. MessageBox, Configuration database, Tracking database.)

- Hosts are a generic container for a pipeline, or an orchestration or an adapter, and therefore becomes a security boundary.

- BizTalk provides the ability to isolate apps within hosts.

- Process isolation ensures code does not impact other hosts.

Public Key Infrastructure (PKI)
- BizTalk can accept encrypted S/MIME v2/v3 messages.
- BizTalk global address book is used for storing partner’s certificate (only public key)
- BizTalk server hosts has its own private key (they use the MY Certificate Store)

Authentication of Senders
- BizTalk will ensure hosts are trusted as well as the messages from these hosts.
- Adapters can be set to authenticate senders if required.
- If authentication is not required, then the sent message will be assigned a Guest ID.
- Establishing authentication of the message is a way to stop DoS attacks.

Note: Currently, the rejected messages cannot be trapped or re-routed.



Building an End-to-End Enterprise Application Integration Solution with BizTalk Server 2004
Terminology
- There is a one to many (1:M) relationship against (receive port: receive location)

- Receive location (m) à receive port (1) à Published to MessageBox à Send port Subscribes to the MessageBox.

- Filter expressions are used against the send port to determine the messages that arrive.

- In BizTalk 2002 the terminology is different (receive function à channels à send ports)

- The receive pipeline is a stateless engine that parses, decrypts, and transforms. (Configured to the receive Port)

- This is connected to the MessageBox (running in SQL Server); this is the PUBLISHING of the message, (you can also promote any specific properties of the message)

- Very symmetrical architecture...

- A business process will SUBSCRIBE to a message...

Adapter framework
- AIC is symmetrical on send and receive ports. (on BizTalk 2002 this was just the send ports)

- Will have an FTP adapter.
- Will have a SQL adapter.

BizTalk orchestration 2004
- Performance will be 10 times that of BizTalk 2002 orchestration.

- Very simple super-scale out.

- Can import/export BPEL orchestrations.

- Nested processes / built in correlation.
BPEL
- Business process contract interoperable across business boundaries... (Sort of like WSDL contract used for Web Services)

- OASIS 105 members expected to get a standard out (some time soon!)

- BPEL is implemented fully with Web Services.


Note: BizTalk 2004 Beta tips. Install sqlxml 3.0 sp1, turn off msmq...

Advanced Business Process Orchestration in BizTalk 2004

Transaction Scopes
Atomic transaction properties.
- Scope is a DTC Transaction.

- Timeout - how long it takes before we decide it’s a failed attempt.
- Retry - do you want to retry transactions.
- Isolation (read committed / seriazable) – database term.
- Synchronized - can only access transactions at the beginning and end of the synchronized transaction. [And not between]

Long Running Transactions.
- Business processes typically last longer than a few seconds.
- Atomic transactions are too resource intensive.

Nested Transactions.
- In BizTalk 2004 it is possible to have transactions inside transactions.
- When inner txn aborts, it does not necessarily mean that outer txn is aborted.

- Atomic transaction compensation is used to undo an atomic transaction within a longer running transaction.

Demo
Drag and drop scope shape component into orchestration designer à Can specify Atomic txn properties à Atomic transactions don't have exceptions since by their very nature they are rolled-back.

Correlation
Correlation is a new feature in BizTalk 2004 to ensure that a message is correctly identified throughout the duration of its orchestration life cycle.

- Routes an incoming document to the appropriate instance of the business process.
- Property promotion. [Within each document, identify a property within a document that is unique within that business process.]

- Correlation sets are a collection of these promoted properties.

- Promoted fields are used for correlation.
- Distinguished fields are used for run-time calculations.

Note: Distinguished fields are different to Promoted fields.

Partner management
[A lot of work has been put into this functionality.]
- When the same business process applies across suppliers with different schemas.

- The business process can be bound to a Role. But doesn't bind the actual end-points.

- Orchestration Designer is used to define which is the best shipper to use at run-time. (i.e. how to make the decision about which supplier to use.)

Human Workflow Services using BizTalk 2004

Human oriented workflow is an area that has never been tackled by Microsoft before.

- Almost anything can be regarded as workflow.

- Ad-hoc tasks, collaborations, non-intrusive workflow, pervasive workflow, spans groupware, LOB, and systems management.

- Human Workflow Services are more focused on ad-hoc Task oriented stuff (fill out forms, organise events, campaigns, review document, RFP, recruiting, etc.)

- Provides Document Tracking displayed visually.

Demo #1
An example of an ad-hoc type workflow à Receive a document à Forward it to a distribution list à Person in distribution list à Opens the document in Word 2003. à Context sensitive task pane (SmartDocs) allows the user to à Delegate the document to another person in dist. List à This person approves the document.

Original sender of the document à Accesses the Workbench à Which shows dynamically a pictorial rendition of the tracking stream (i.e. what’s been happening to that document)

Demo #2
Utility called "sn.exe" that creates public/private key-pair (e.g. sn -k .snk) [creates a key-pair.]

Drag and drop "GAC Attribute" to AssemblyInfo.* à specify key-pair filename à then build.