Pros and cons of an open source data catalog

by
Kira Lenz
March 18, 2022

In software development, open source technologies and open source components have been used for quite some time and are almost indispensable. When it comes to the use of complete open source solutions in larger, non-tech-based companies, the picture is currently quite different. There, the view has long been held that a large company needs a large enterprise solution and that an open source catalog, for example, cannot meet the requirements. With increasingly powerful offerings and a steadily growing number of companies opting for them after all, a slow change is now beginning. 

Evaluating whether an open source data catalog is a good solution in an individual situation should be done very thoroughly. Certain features such as the existence of support or available connectors can, of course, already be a knock-out criterion. However, if the basic criteria are met by both an open source catalog and an enterprise catalog, individual criteria must be evaluated. For this purpose, we strongly recommend prioritization that takes all user groups into account.  Whether the possibility of individual configuration is more important for an example company than additional effort in implementation and maintenance must be decided on a case-by-case basis. 

We have summarized the most common pros and cons that we have considered and weighed up together with partners and customers here for your orientation. 

Plus points

  • Flexibility: The customization options are usually significantly higher and uncomplicated due to open interfaces.
  • Agility: Thanks to active communities, the latest features are available quickly
  • Transparency: Due to the free availability, the code is completely visible and problems can be discovered quickly. 
  • Price: The code is free of charge and you have full cost control through your own hosting (BUT: Contract development may be necessary and should definitely be included in the cost planning)
  • Scalability: With a state-of-the-art deployment, there are no technical limits to scalability, nor are there any user restrictions on the part of the provider.
  • Quality: An active community and correspondingly robust review processes ensure a high standard of features.
  • Vendor independence: As a user, you are not dependent on whether the vendor supports the tools you use in the future, and you also retain the ability to design, expand, and change your own infrastructure in the future, free of lock-in restrictions.

Challenges

  • Expertise: a higher level of expertise in ops and development teams is required to build, customize, maintain and use it 
  • Support: there is no "real" support. However, active communities are often very helpful and external service providers can be hired to provide the appropriate services 
  • Interface: Open source solutions often have simpler UIs than commercial solutions and may be a bit more challenging for technical users
  • Features: Large commercial providers sometimes offer even more connectors to complete enterprise platforms and more simplifying features such as drag-and-drop functions
  • Time investment: Independent implementation and maintenance are of course associated with time investments that are not required when hosted by a provider.

In our experience, there are some typical use cases for which an open source catalog is very suitable. Companies should definitely consider one if they are

  • Are in the process of developing or modernizing their data architecture and want to flexibly test concepts
  • Have complex-architectured data structures and need a lot of flexibility to connect the individual objects
  • Have advanced architectures and want to implement more complex use cases that involve a lot of automation

In summary, open source data catalogs offer many opportunities, but they also have their drawbacks. By clearly identifying the use cases and prioritizing the various pros and cons, organizations can make an informed decision regarding suitability. 

If you have any questions, please feel free to contact us!

Are you interested in more information? Let's talk about your current topics and challenges!
Contact