NOTE: A Big Analytics Announcement (Visual Analytics)

Following March 6th's announcement of SAS support for Hadoop (for massively parallel access to massive volumes of data) came March 22nd's announcement of SAS Visual Analytics, a new, in-memory tool with a highly visual interface.

The Visual Analytics solution contains an in-memory analytics engine that will eventually extend across a large amount of SAS's software product portfolio, including publish-to-mobile capability and graphic visualization.

Bringing recent announcements together and creating synergy, a core component of SAS Visual Analytics, the SAS LASR Analytic Server, uses Hadoop (embedded Hadoop Distributed File System) as local storage at the server for fault tolerance.

SAS Visual Analytics includes:

• SAS LASR Analytic Server - clients communicate with SAS LASR Analytic Server for calculations on the data resident in-memory
• The Hub - a central location to launch the various elements of SAS Visual Analytics
• Mobile - a tool for viewing reports, connecting to servers and downloading information on the go
• Explorer - an ad hoc data discovery and visualization tool to explore and analyze data
• Designer - used to create standard and custom reports and dashboards
• Environment Administration - used by administrators to manage users, security and data
• Server components run on Red Hat or SUSE Linux, and the mobile client is available for the iPad.
  Android support is expected to follow.

SAS's demo video for Visual Analytics hints at some of its capabilities without revealing too much of the detail. I'm looking forward to seeing a lot more detail at next month's SAS Global Forum. Will Visual Analytics knock spots off of existing in-memory solutions such as Qlikview, Spotfire, and Oracle Exalytics? Only time will tell, but it sure looks promising.

Combined with the fact that SAS was recently named as a leader for its data extraction, transformation and loading (ETL) technology in "The Forrester Wave: Enterprise ETL" by Forrester Research, and in the Leaders quadrant for its business intelligence (BI) technology in Gartner's Magic Quadrant for Business Intelligence Platforms, SAS has never looked stronger as an end-to-end solution.

NOTE: Keep Your WORK Library Working

Steve Overton recently posted a tip on the excellent BI-Notes blog about maintaining space within your WORK library. The WORK library is a key resource and is typically shared by all SAS users, so its smooth operation is crucial for maintaining a reliable service.

Steve offered a unix/linux shell script to winkle-out old files that might otherwise be missed by SAS's cleanwork utility. It's clearly a popular topic and Steve's post received a number of responses, so I thought I'd offer my own small tip.

More than one of my clients applies a quota limit to all users on their use of the /saswork file system. By setting the quota to a suitable proportion of the whole amount of space available to everybody, the quota prevents one user from hogging all of the work space and causing other users' programs to fail. For example, if you have 1TB of WORK space, you might set the quota to 500GB. It's not a silver bullet, but it works well in practice.

The Geek Stuff provides a neat "how to" on the subject for unix/linux. I imagine that a similar technique is possible on Windows too, but I haven't the experience to say so for sure.

NOTE: A Big Data Announcement (Hadoop)

There was a time when SAS practitioners merely needed to have knowledge of SAS DATA Step and PROC syntax plus a smattering of good data practices and IT practices. Over the years, SAS/ACCESS products gave access to data in other systems without the SAS practitioner needing to know anything but access credentials for those other systems. In more recent years, SAS version 9 has, on the one hand, introduced a range of SAS clients that allow the user to focus on their data knowledge and analytical skills rather than their SAS coding expertise, whilst, on the other hand, SAS version 9 has introduced (necessary) complexity to the architecture incorporating multiple types of SAS servers, comprehension of TCP/IP ports, third-party components such as web servers, plus the Platform suite of tools (from LSF job scheduling to Grid management).

Into this smorgasbord of clients and architecture, SAS added some fascinating new components earlier this month. I was interested in the new features and capabilities, but I also wanted to understand whether this brought more or less complexity for the SAS user and/or those charged with SAS platform support.

So, firstly, what was the announcement? Well, on March 6th, SAS announced the introduction of Hadoop support as part of Enterprise DI Server. Hadoop, in a nutshell, is an Apache, open source, product that provides massively parallel access to massive volumes of data. Significant users and supporters of Hadoop include Amazon, EBay, Facebook, Google, IBM, Macy's, Twitter, and Yahoo. No matter how you look at it, this is a big announcement if you are into big data; and if you're not yet into big data, maybe you soon will be.

There are multiple technologies associated with Hadoop, and SAS seems to have covered them all. For instance,

• SAS/ACCESS will provide seamless and transparent data access to Hadoop (via HiveQL). Users can access Hive tables as if they were native SAS data sets.
• PROC SQL will provide the ability to execute explicit HiveQL commands in Hadoop

• SAS will help execute Hadoop functionality with Base SAS by enabling MapReduce programming, scripting support and the execution of HDFS commands from within the SAS environment. This will complement the capability that SAS/ACCESS provides for Hive by extending support for Pig, MapReduce and HDFS commands. [yes, I did copy the text from the SAS web site; no, I don't (yet) fully understand all of the terms!]
• DI Studio will include Hadoop-specific transforms for extracting and transforming data

I'm slowly getting up-to-speed with this stuff myself, and I'll certainly be on the lookout for knowledge at SAS Global Forum next month. In the meantime, I found a couple of blog posts by Mark Troester most informative [1, 2].

The software release is certainly getting large amounts of positive comment from the technology media. The article in Information Week is just one example, singing SAS's praises.

And so, to return to my original question: has this announcement brought more or less complexity for the SAS user and/or those charged with SAS platform support? It seems clear that the SAS platform architect will need to understand Hadoop concepts, and that will require additional skills and knowledge. On the other hand, it sounds like SAS clients will do a great job of allowing the user to focus on their data and their analytical processes rather than learn new Hadoop-specific technical skills. On balance, I'd say that's the right compromise.

NOTE: SGF 2012 Early-Bird Registration Ends March 19th

If you haven't already registered for SAS Global Forum 2012 (SGF) you need to hurry-up because the early-bird discounts will end on March 19th.

If you're not convinced of the value of attending, check-out last year's best contributed papers to see the quality and breadth of what's on offer. That's without taking account of the demo area and the ability to network with colleagues and peers.

NOTE: B&D to be Acquired by Sopra

One of the UK's largest independant providers of SAS skills, Business & Decision UK, is to be acquired by the Sopra Group. Jointly announced with the same press release (B&D, Sopra) on the 13th February, the transaction is expected to be completed in the next few weeks, subject to regulatory conditions.

You may be able to remember SPS, one of the very first UK-based SAS consultancies, established by Peter Bailey. B&D, headquartered in France, acquired SPS some time ago. Whilst the likes of Amadeus and Base3 continue to provide focused SAS services in the UK, it seems that the trend is towards larger suppliers with a broader base of technology and integration skills. What do you think?

Sopra Group is also headquartered in France. It's results for 2011 show revenue of 1bn euros, 13,000 staff and a focus on IT services and software development.