The world of data warehousing is shifting into a digital, agile and fast-paced era and development of SQL is nothing new, there is a constant increase in the demands to generate reports quickly and help in guiding data-driven decisions. However, this is creating a lot of pressure on professionals to create consumable server data for the growing demand with several layers of technical expertise and access credentials. There is also a pressure to allow self-serve BI in a secured, easy and controlled way.
Here are few points to consider:
- “Victory in agility” means demands on many people with access to the same insights at the same time
- A company having bottlenecks to prepare, process, acquire and serve data may lead to prime decisions made on decisions
- The data is available in volumes of petabytes, and the businesses have a high demand for reliability and availability
- The new data types and the existing data sets need to be joined quickly
- The data types to be queried but the data mainly resides in separate silos
- The quick change in business needs fine-tuning or changes in the existing questions and reports too
Security integrations, IT budgets, scale, efficiency, ease of access, etc., are existing data warehousing entities stressed by these demands and trends. Serving the business efficiently is another stress reflected on the SQL developers and end users. Cloudera Data Warehouse – a modern data warehousing compound launched by Cloudera is designed to help optimize the mentioned and majority stress elements.
It is easy to operate SQL Developer Workbench: Hue, widely adopted and specifically designed to meet the needs of skilled modern worker.
Benefits of Hue: SQL developers Workbench
- You need not worry where the data lies as hue helps navigate to the correct data be the data in cloud or on-prem
- 100% expertise on your data is not required as hue with guiding intelligence helps choose right indexes, files and tables to work with accurately and confidently
- You need not be a backend SQL expert as hue suggest optimizations, warns about bad query designs, allows the query to execute wherever compute is available (i.e. in cloud or on-prem)
- Learning new skill is not required as hue gives familiar and same experience of SQL Workers on the Cloudera’s platform.
Here are some highlights of the latest Cloudera Enterprise (C6) realized, a Hue 4.0 is enabled here by default that aims to simplify and expedite common job for modern SQL user:
Data ingestion
Data is ingested via data engineering pipelines, to satisfy customer needs, Cloudera comes with many integrated tools like Kafka, Flume, cloud service option, Flume, and optimized partner solutions from Streamsets and Informatica. Some data i.e. few tables in an external DB and few files on S3 are common to lay for a SQL user. Bringing in files and tables is well guided and easy via Hue, it also connects to backends like S3, ADLS, MySQL, etc., to streamline the jobs of important ingesting extra data sets.
Data Exploration and Discovery
To find the right data for a large organization to work, several ways are implemented to expedite discovery or data and exploration. It is easy to search for namespaces, tables, databases, collections, views, and even files for the customer in a C6 simplified data browser. Additionally, it is easy to sample and preview the data for validating that you have relevant assets for working.
Cloudera Navigator tool (that is a part of Cloudera Data Warehousing portfolio, helps with lineage, auditing, and catalogs) with distinctive integration allows us to see the most commonly utilized tables (crowdsourced). It is helpful information that indicates which table to work with in case you are not sure, it cuts the discovery phase by several hours. The search function has tags so that you can easily find the datasets at your next visit – once you find the correct datasets.
Query Designs
The SQL users often have to revisit the coding, to avoid this we provide the best of both the worlds – to save queries and see query history, to fasten the procedure of getting started. With this SQL workbench, the users can design queries iteratively. In case of an arise of a new business question, you can conveniently start from the saved query, copies/sections of previously executed queries to start new query design.
There are many helpful tools throughout this query design procedure to assist, an auto-complete functionality to allow typing both table names and commands, it helps dynamically with suggestions too. Auto-complete speeds up the process by 10x and it also proposes tables depended on frequent use.
Optimization and Visualization
Navigator Optimizer is another tool in this Data Warehousing portfolio to allow users to access the SQL statements potency quickly. This tool warns on non-good query design and suggests optimizations to achieve better performance. Using the Navigator Optimizer, a proactive SQL adds flexibility to the whole service, it prevents the memory-intensive query to hold services that can be utilized for query execution and other users.
The results obtained (be it from a structured query system, impala in this case or unstructured query system, Solr), can be viewed easily in a simple drag-drop dashboard. It helps the workers visually and quickly share their insights.
Sharing
To share the results with other customers or audience directly, you can find simple methods of posting created queries or views to GitHub. The border between ML-focused data scientist and traditional SQL analyst is slightly blurring, thus, a collaboration between them is important. Sharing these assets between a tool and these groups like SQL Developer Workbench and Cloudera’s Knowledge Worker enabling a seamless transition is also important.
Conclusion
This trend of modern data warehousing services is currently in a boom, it nothing but collaboration and self-service at multiple stages. It is being enabled at the query level with Hue. The user experience of Cloudera’s SQL is based on valuable and unique integrations, this SQL Workbench permits users to optimize, troubleshoot and design queries iteratively and speed up the overall process. Hue, carries your work, fastening the SQL users to start new projects.
The SQL Developer Workbench which comes with the Cloudera’s Modern Data Warehouse is an exploration, development, leading discovery, and collaboration tool today. Hue is easy to use and key choice of modern data warehouse users.
