Introducing The Namara Enterprise Edition Tech Stack

Shared by  Chris Sandison  on October 15, 2020

in Namara, DevNotes, Engineering, Namara Enterprise Edition

Hi there, it’s Chris! 

I’m the Director of Engineering at ThinkData Works. I'm eager to share with you our latest in development updates for the Namara platform and to provide some context on the direction that we’re going here at ThinkData.

 
Our team is constantly operating in beta, which means that we believe that there's room to improve on every component of the platform. Over the past six years, we have gone through numerous iterations to ensure that our tech is always one step ahead (at least). Our latest iteration was a massive undertaking, but we couldn't be happier with the result: Namara Enterprise Edition. 

The basics – what is Namara?

The Namara platform is designed to let organizations use more data with less overhead and higher confidence. Our goal since day one has been to facilitate easy access to data – as much as possible, as clean as possible, for as many people as possible. It's a data marketplace as well as a set of robust data tools that allow for intelligent ingestion, intuitive transformation, secure collaboration, and simple integration.

In building, maintaining, and improving Namara, we focused on solving for data variety and providing streamlined access to clean data. The launch of Namara Enterprise Edition is an evolution of this core idea, building on what we've learned from our users about how they want to connect to the information powering their decisions. It is now more capable than ever of letting our users get the most out of every piece of data they have access to.

Here's what the Namara Platform is comprised of:

Namara Platform Illustration

Now that you have a picture of the platform, I want to introduce you to a few of the talented engineers that have made major contributions to creating the new version of Namara. Each of them brings a specialized understanding to every aspect of our tools and services.

ETL trained on the most data

Publicly available data is a moving target – we needed to develop an ingestion pipeline designed to pull in data while solving for data variety. Since public data is often released in non-traditional and proprietary formats, we developed our ingest pipeline to be fully customizable for the end user, enabling import configurations to neutralize data variety at the source.

“Our ingest pipeline provides more configuration at the user level than any other. We see many new variety cases daily, which are solvable by making simple changes to a YAML file.”

– Matt, Ingest Team Lead

New Features:

  • Zero-penalty ingestion
  • 25x speeds over comparable ingestion tools
  • custom YAML-based import configurations
  • easy file selection within compressed archives

Read more about Ingest here

Manage incoming data streams from one place

Imports on Namara provide a place for you to directly connect to external data providers. Boost productivity by saving time switching tools and making different products compatible with each other.

“Importing data into Namara is designed to be truly agnostic when it comes to the variety of sources our clients use. Every data source has widely different features, interfaces, and behaviour; Namara standardizes it all, so data scientists just need to learn one tool instead of dozens.”
– Justin, API Team Lead 

New Features:

  • access previous data snapshots with versions and tags
  • configure Namara to query directly from your own warehouses - no Ingesting required, so the data stays where it is!
  • source and credential management
  • drag-and-drop data importing
  • advanced import features
  • status monitoring
  • import scheduling

For a live demo of Namara connecting to public data, please reach out.

Explore, share, and integrate any dataset

The dataset table is your home for navigating large datasets in-browser. With key filtering and sorting features, you can quickly evaluate and explore a catalog of data. With one click, get mapped geospatial data, dataset metadata, licensing information, or integration into your own environments, applications, and BI platforms.

“We’ve vastly improved the speed and performance of the table thanks to a dedicated, underlying query service. This means users can start working with the data they need faster than ever.”
– Bronek, Staff Engineer 

Improved Features:

  • Table view
  • Sorting and filtering features
  • API info tab
  • Map 
  • Integrate with preferred native environment

Explore our catalog of data and search for better data here

Detailed and intuitive role management

Customize organizational hierarchy, empower other members with management capabilities, and fine-tune dataset visibility. Users can be given access to the features they need and restricted from accessing the features they don't.

"By evolving our permission model to a role-based access control (RBAC) system we now provide a ‘single pane of glass’ to see what users have access to" - Maha Islomova, Marketplace Team Lead 

New Features:

  • Organization membership management
  • Manage collaboration with Groups
  • Role-based access control
  • More robust user invitations
  • Dataset sharing

Why is role management important? Read more here

Filter, join, & enrich data using familiar queries

NiQL is SQL for the Namara catalog – any user can explore, experiment, and deliver new products built on live data. It has all of the core functionality of standard SQL, enabling users to filter, aggregate, and join different datasets. Send NiQL queries over the API or using any of our client libraries to pull live data into your environment or applications.

“If the Namara API is the single point of access for any dataset, then NiQL is our standard of access. Our goal for NiQL was to create a tool that would feel familiar to any developer or data scientist, and would allow them to integrate Namara into any environment or application.” 
– Chris Sandison 

New Features:
  • Standard SQL behaviour coverage
  • PostGIS features

Read more about NiQL here

Bring Namara data into any environment

The re-imagined Namara platform was built for portability, to be a master data source that integrates with any and all that tools data professionals use. Our collection of client libraries, plugins, and integration tools will continue to grow to meet the needs of developers or data scientists.

"From Pandas and Jupyter Notebooks to Tableau, we want to make sure that data teams can integrate data from Namara into any solution seamlessly.”
– Armin, Staff Engineer

New Features:

  • Platform SDKs in GO
  • Javascript
  • Ruby
  • Python
  • Java and R
  • Pandas DataFrames
  • JDBC/ODBC connector

Read more about integrations here.

That's a wrap – for now

I hope you enjoyed getting to know a bit more about the Namara tech stack and hearing from some of the team that powers our tech. We believe that transparency is important, so as an organization, we'll be sharing regular development updates to showcase our latest iterations and advancements.

We look forward to introducing you to the Namara Enterprise platform and guiding you through the new and improved tools and features available. If you haven’t used Namara before and want an introduction, let us know – we’d be more than happy to walk you through the platform and answer any questions you have.



Want to learn more about
 aligning your business and data strategy? Browse the largest catalogue of solution-ready data or book time with our experts to determine how ThinkData’s tech can advance your projects.


Never miss an update from ThinkData: