Databricks Launches 'Apps' for Rapid Development of AI and Data Applications

Databricks' New Offering Promises Speedier Analytical, AI Application Development

With the new Databricks Apps partners and customers can rapidly build and deploy native applications for the Databricks Data Intelligence Platform that tap into the system's data and leverages its data security and governance capabilities. Databricks is launching a public preview of Databricks Apps, a new set of development capabilities that the company says provides a fast way to natively build and deploy internal data-intensive analytical and AI applications directly on the Databricks Data Intelligence Platform. The development services are particularly geared toward developing custom software such as AI applications, analytical applications, data visualization dashboards, self-service analytics capabilities and data quality monitoring software. "Our mission is to democratize data and AI. And as part of the data intelligence platform, we're building a platform that lets every customer get value from their data and from their investment," said Shanku Niyogi, Databricks vice president of product management, in an interview with CRN. [Related: Databricks CEO Ghodsi: Systems Integrator Partners Are Key To Winning 'The AI Revolution'] Businesses and organizations today are developing AI and data-heavy applications to take advantage of the growing volumes of data within Databricks, Snowflake and other data platforms. But Niyogi said that presents a number of challenges including complex development processes, integrating the applications with data sources and data pipelines, and securing and governing data as it flows into applications and AI models. "What we're basically doing with Databricks Apps is we've built a super-simple way to get through those obstacles when you're building an app," Niyogi said. "Databricks apps are very easy to build. We've tried to take an open approach so you're not locked into one framework or one tool." The initiative is part of the company's overall mission of "democratizing data" by helping customers generate value from data that's locked up in data lake systems, Niyogi said. "It's really to help customers unlock the value they have from their data and their AI in the simplest way possible and be able to drive lots of consumption in the [Databricks] platform." Databricks Apps is initially focused on Python, the top programming language for data-intensive applications. Databricks Apps enables developers to build apps natively in Databricks using such tools as Visual Studio Code and PyCharm and popular Python frameworks, such as Dash, Shiny, Gradio, Streamlit and Flask. "We have a set of built-in templates," Niyogi told CRN. "If you've got an existing app, you can easily bring that to the platform as well. And we support all the authoring tools as well that most developers are familiar with." Databricks Apps also makes it possible to incorporate AI components within applications, making it possible for developers to call specific AI models when they need more flexibility, according to the Databricks blog announcing Databricks Apps. Niyogi said Databricks Apps are "super easy" to deploy, provision and host directly out of Databricks through the use of automated serverless compute features and by building automation into development tools and processes. And the applications are easy to secure because the applications, data and models all run fully within Databricks, Niyogi said. "You never leave Databricks, unless you really want to. You can run the entire self-contained app and all of its data and AI dependencies directly in your Databricks workspace." That workspace has security controls including authentication and permissions, Niyogi said, and the applications are fully governed by the Databricks Unity Catalog centralized data governance system. Databricks' channel partners, including some 3,300 consulting and systems integration partners and 750 ISV partners, also will benefit from the new Databricks Apps, Niyogi said. "We've been building a pretty extensive kind of partner ecosystem," the executive said. "So when we started [Databricks Apps], we actually started by talking to a bunch of partners. We figured we could build a simple story for developers, but what we really wanted to do is unlock the ecosystem with this. So we went and talked to a number of partners, people in verticals that were building applications with data [for] financial services, healthcare, marketing and consumer insights." Noting that his company operates the Databricks Marketplace, Niyogi said Databricks Apps greatly benefits the partners who develop and sell pre-packaged applications, data products, and other AI and analytical assets. "And because these apps run in the customer's workspace, it opens up a lot of ability to scale those partner efforts," Niyogi said. "We see huge potential here for our partners to be able to reduce friction in their work with customers by building apps to help them scale."

Databricks now lets developers create AI apps in 5 minutes: Here's how

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More Databricks just made app development a piece of cake. The Ali Ghodsi-led company announced Databricks Apps, a capability that allows enterprise developers to quickly build production-ready data and AI applications in a matter of clicks. Available in public preview today, the service provides users with a template-based experience, where they can connect relevant data and frameworks of choice into a fully functional app that could run within their respective Databricks environment. According to the company, it can be used to create and deploy a secure app in as little as five minutes. The announcement comes at a time when enterprises, despite being bullish on the potential of data-driven applications, continue to struggle with the operational hassle of the entire development cycle, right from provisioning the right infrastructure to ensuring security and access control of the developed app. Much like Snowflake, Databricks has long provided its customers the ability to build apps powered by their data hosted on the company's platform. Users can already build applications such as interactive dashboards to delve into specific insights or sophisticated AI-driven systems like chatbots or fraud detection programs. However, no matter what one chooses to develop, the process of bringing a reliable app to production in a secure and governed manner is not an easy one. The developers have to go beyond writing the app to handle several critical aspects of the development pipeline, right from provisioning and managing infrastructure and ensuring data governance and compliance to manually bolting integrations for access controls and defining who could use the app and who could not. This often makes the whole development process complex and time-consuming. "App authors had to become familiar with container hosting technologies, implement single sign-on authentication, configure service principals and OAuth, and configure networking. The apps they created relied on integrations that were brittle and difficult to manage," Shanku Niyogi, the VP of product management at Databricks, tells VentureBeat. To change this, the company is now bringing everything to one place with the new Databricks Apps experience. With this offering, all a user has to do is select a Python framework from a set of options (Streamlit/Dash/Gradio/Flask), a template of the type of app they want to develop (chatbot or data visualization app) and configure a few basic settings, including those for mapping resources (like data warehouses or LLMs) and defining permissions. Once the basic setup is done, the app is deployed to the user's Databricks environment, allowing them to use it themselves or share it with others in the team. When others log in, the app automatically prompts them with single sign-on authentication. Further, if needed, the developer will also get the option to customize the developed app and test their app code in their preferred IDE (integrated development environment). On the backend, Niyogi explained, the service provisions serverless compute to run the app, ensuring not only faster deployment but also that the data does not leave the Databricks environment. "Each app is fortified with robust security measures for seamless and secure user access. Plus, the integration with Unity Catalog provides comprehensive data governance and management capabilities, while the apps inherit the networking protections of your workspace, ensuring a multi-layered security approach for your sensitive data and applications," he explained. At this stage, Databricks Apps only supports Python frameworks. However, Niyogi noted that the company is working to expand to more tools, languages and frameworks, making secure app creation easier for everyone. "We've started with Python, the #1 language for data. Anyone familiar with a Python framework can write their app in code, and anyone with an existing app can onboard it into Databricks Apps easily. We support any Python IDE. We are working with ISV partners to enable their tools to support Databricks Apps, and add support for other languages and frameworks," he added. Some 50 enterprises have already tested Databricks Apps in beta, including Addi, E.ON Digital Technology, SAE International, Plotly and Posit. With the public preview launching today, the number is expected to grow in the coming months. Notably, Snowflake, Databricks' biggest competitor, also has a low-code way to help enterprises develop and deploy data and AI apps. However, Databricks claims to distinguish itself with a more flexible and interoperable approach. "Databricks Apps supports Dash, Gradio, Flask, and Shiny as well as Streamlit, and supports more versions of Streamlit than Snowflake does. Developers can also use their choice of tools to build apps. We will continue to build on this flexible approach, adding support for more languages, frameworks and tools," Niyogi pointed out.

Introducing Databricks Apps : The fastest and most secure way to build data and AI applications

Databricks Apps, a new way to build and deploy internal data and AI applications, is now available in Public Preview on AWS and Azure.Ideal use cases include data visualization, AI applications, self-service analytics, and data quality monitoring.It supports Dash, Shiny, Grado, Streamlit, and Flask app development frameworks.Automatic provisioning of serverless compute provides easy app deployment.Built-in governance with Unity Catalog, and secure user authentication through OIDC/OAuth 2.0 and SSO. We are thrilled to announce the Public Preview of Databricks Apps, the fastest way for Data and AI teams to build and deploy internal applications directly on the Databricks Data Intelligence Platform. Databricks Apps enables developers to build apps natively in Databricks with popular frameworks, such as Dash, Shiny, Gradio, Streamlit, and Flask. One of the key advantages of Databricks Apps is the ability to create data applications tailored for non-technical users using code instead of SQL. This opens up new possibilities for making complex data insights accessible to a wider audience within organizations. For instance, a marketing team can leverage Databricks Apps to create customized dashboards that visualize campaign performance metrics, allowing team members without technical backgrounds to interpret and act on the data easily. Moreover, Databricks Apps can incorporate AI components, enabling developers to call specific AI models when they need more flexibility. This integration of AI capabilities allows for the creation of sophisticated applications that can perform tasks such as sentiment analysis on customer feedback or predictive modeling for sales forecasts, further enhancing the value of data insights for non-technical users. Once built, the apps are deployed and fully managed directly in Databricks, saving teams the effort to configure and manage infrastructure. These apps are fully governed, respecting the data access controls already configured in Unity Catalog, and controlling the distribution to users using the same unified governance model. With Databricks Apps, organizations can harness the full potential of their data and AI investments by creating custom applications that run seamlessly within their Databricks environment. The challenge in building data applications In today's data-driven world, organizations are seeking ways to extract more value from their data assets. However, building and deploying internal data applications has traditionally been a complex and time-consuming process. Developers need to spend time on infrastructure management instead of focusing on app development. Data governance and compliance require manual implementation of access controls. Additionally, app sharing and permissions are managed separately from other data assets, creating a disjointed governance experience. Databricks Apps: Quickly build secure data applications Databricks Apps addresses these challenges head-on, providing a powerful yet simple experience for building internal data applications. By adopting Databricks Apps, organizations can unlock numerous advantages: Simple to Build Databricks Apps helps you build apps that run directly within your Databricks environment or with tools, such as Visual Studio Code and PyCharm, ensuring seamless access to your data and AI models. With Databricks Apps, data scientists and engineers can rapidly build and iterate on apps using familiar Python frameworks such as Dash, Gradio, and Streamlit. You can also choose from pre-built Python templates that allow you to quickly build flexible apps. "Databricks Apps helped me turn my RAG proof of concept into a polished and branded application. We built a RAG system to answer user questions by utilizing our company's extensive knowledge base." Heather Gomer, SAE International Production Ready and Automated Deployment Databricks Apps does not require developers to build additional infrastructure. Apps run on automatically provisioned serverless compute, allowing deployment with ease. Databricks Apps also embraces industry-leading development practices, offering seamless integration with your preferred workflow. Whether you choose to work directly within the Databricks workspace or leverage your favorite IDE, you'll benefit from support for Git version control and CI/CD pipelines, ensuring your internal apps are production-ready. "The seamless integration of Databricks Apps into our DevOps processes enables us to quickly demonstrate and test new features with users while also providing a secure, production-ready front end for the internal application -- all without needing additional infrastructure." Lukas Heidegger, E.ON Digital Technology Built-in Governance With Databricks Apps, data only leaves your Databricks environment if you choose to share it. Each app is fortified with robust security measures, including granular access control to ensure precise data permissions, automatically managed service principals for secure application-to-application communication, and automatic user authentication leveraging OIDC/OAuth 2.0 and SSO for seamless and secure user access. Furthermore, integrating Unity Catalog's lineage capabilities provides comprehensive visibility into your applications' data origins, transformations, and usage, enhancing data traceability and compliance. This integrated approach ensures that your data applications adhere to organizational policies and regulatory requirements while also facilitating data discovery and promoting data use across teams. "By using Databricks Apps, we saved many rounds with the security and infrastructure team and were able to instantly share our app with stakeholders in production." Cesar Augusto Charalla Olazo, Addi Common App Patterns Databricks Apps can be used to build a variety of internal applications, including: Custom Data Visualization: Create dynamic, data-driven visualizations that allow business users to explore and analyze data in real-time.AI Apps: Develop applications that leverage machine learning models for tasks like predictive maintenance, customer segmentation, or fraud detection.Self-Service Analytics: Enable business users to perform complex analyses through user-friendly interfaces, reducing the burden on data teamsData Quality Monitors: Build custom tools to track and improve data quality. "We realized the user-facing data interface of our Health, Safety & Environment Intelligence Platform fully with Databricks Apps. We now host a Streamlit dashboard, featuring a semantic search tool, alongside various other dashboards." Lukas Heidegger, E.ON Digital Technology Posit (Developer Tools Partner of the Year 2024) has long believed in the power of creating applications using code-first tools to help organizations derive insights from their data. This belief inspired the creation of Shiny for R, Shiny for Python, and Posit Connect, as well as our collaboration with Databricks Apps to support a variety of applications. We look forward to our continued partnership with Databricks to make code-first tools as ubiquitous and accessible as possible. - Tareef Kawaf, CEO, Posit "Plotly (Databricks' 2024 "Customer Impact" Partner of the Year) applauds the introduction of Databricks Apps and its enablement of analytics professionals to serve business users. Databricks Apps offers an easy way for Databricks customers to begin journeys toward using Databricks with Plotly's Dash open-source library for the range of sophisticated production-grade data app use cases that Plotly is known for via its Dash Enterprise offering." - Dave Gibbon, Sr. Director - Strategic Partnerships at Plotly Getting Started with Databricks Apps Databricks Apps is now available for all workspaces in the supported regions. To write your first app, go to + New and click Apps. Follow the instructions on the screen. Make changes using your favorite source code editor, and deploy! See the documentation (regional availability: AWS, Azure) for more information on all the features. We can't wait to see what you will build with Databricks Apps, where you can start building powerful, data-driven applications today and unlock new possibilities for your organization.