Big Data moves into cloud security

A factor that was noticeable at the InfoSec exhibition and conference in London, earlier this year, was the lack of any evidence that Big Data was seen as part of the cloud security mix – indeed, any security application of the technology. But that may be about to change with the introduction by US vendor, Radiant Logic, of HDAP, which it describes as a highly scalable LDAP Directory, driven by Big Data and search technology, and targeting identity, access and data management roles.

These are obvious targets for the application of Big Data analytical tools, particularly in relation to cloud services use, for they are key areas where security-by-policy is most likely to be used. Here, a business can determine its policy on who has access to data, how they authenticate their identity, when they can access it, where they can access it, and what they can do with it.

That, of course, is easy enough to write as a set of ideals, but to make it happen requires a range of new tools, coupled with new ways of thinking. The tools will need to be fast because they will have to react in real time in order to trap any improper or malicious activity. They will also need to be fed by a vast range of both hardware and software monitoring devices, so the quantity of data will be prodigious.

The new ways of thinking about cloud security will include the development and imposition of business related policies on the activities conducted in the cloud. This will also involve creating new ways for data to protect its integrity, up to and including self-destruction to prevent data loss if necessary.

With that in mind it is interesting to see early steps along this route now being taken. Radiant Logic has introduced HDAP, which it claims is the world’s first commercial solution for distributed storage and processing for enterprise identity management. Based on Hadoop, this is a highly scalableversion of the LDAP directory.

HDAP will form part of the company’s upcoming RadiantOne 7.0 virtualisation release, which is currently available in Beta form and is due for release during the fourth quarter of this year. It is intended to allow companies to radically scale their access and throughput, and is said to be the first highly scalable and secure directory that’s based on big data and search technology.

Identity management is coming under increasing threat as cloud services, coupled with much wider access from mobile devices of all types, become prevalent. To keep up with authentication and authorisation demands, while tapping into greater use of personalisation and recommendation engines, companies need a richer view of their identity, along with better performance and greater flexibility.

The traditional LDAP server has not managed to keep pace with such developments over the past decade, neither in terms of scalability nor in richness of defence capabilities. This increasingly impacts the capabilities of current IAM infrastructures, and makes it difficult to meet the needs of today’s mobile and cloud-driven workforce—or the customers they serve.

The growth of cloud-enabled environments and the limitations of LDAP have also made it all but impossible to increase access speed and reliability. Breaking through that barrier is target Radiant Logic has set for HDAP, which couples Hadoop and search technology with LDAP to open up  performance possibilities for IdM and data management.The company is also aware that other security possibilities may then follow.

According to Dieter Schuller, Radiant Logic’s VP of business development,  the company’s customers use RadiantOne to streamliner identity and access infrastructures and speeds deployments. It is also used to integrate more attributes from across infrastructures and publish new views into into identity data. But it had still lacked a way to store dynamic and complex views of identity. Such a solution needed to scale on demand, and support smart search and analysis at a scale that till now had been unachievable for the enterprise. That, he suggested, had now been achieved with HDAP.

“Finally, there’s a robust security solution that’s built on a well-known model that also leverages the best of today’s technology,” Schuller said. “Now companies can more easily integrate with cloud applications, federate with partners, and enrich their authorisation and access policies.”

When the new HDAP-enabled RadiantOne 7.0 system is launched, users will be able to handle high access demand without slowing the access process. Currently in beta, a single node of the new HDAP store can handle 50,000 queries per second, and because the image is replicated across all nodes, load balancing allows the system to increase the aggregated throughput in a near-linear fashion.

As well as providing speed and performance improvements, HDAP also leverages the company’s virtualisation layer, allowing it to serve as storage for synchronisation.

“By deploying a full LDAP directory on top of a cluster of Hadoop/ZooKeeper nodes, the new RadiantOne offers extremely tight levels of replication between nodes, ensuring that the image of the enterprise’s entire identity infrastructure is always kept up to date across every node,” said Radiant Logic CTO Claude Samuelson.

The notion of finding new ways to exploit security technology in proactive ways that enhance the value proposition of a business has already occurred to Radiant Logic. RadiantOne can already virtualise and integrate a wide range of structured data from application silos, making that information more available and relevant for the enterprise. The HDAP-enabled version is expected to extend that capability considerably.

“Thanks to our patented context representation for structured data, RadiantOne can extract the contextual relationships between once-siloed information and represent all that essential context using plain English sentences, readable by both man and machine,” says Radiant CEO Michel Prompt.

“We were using an LDAP engine to store and search all this information. But now, with this latest breakthrough, the directory at the heart of IdM is becoming a system to publish contextual information from enterprise applications, making it searchable in the same way that we search the Internet—by keyword. As we adapt more and more context-aware devices, we will also need high-speed context servers. Combining Hadoop and search technology with LDAP and context changes the game for IdM, making the future of identity much richer and more responsive to demand. In fact, with HDAP and context, we’re bringing IdM and security back where they belong—in the center of data management.”

.