Why NoSQL (vs SQL)

NoSQL databases that enable storing unstructured and heterogeneous data at scale have gained in popularity. For most developers, relational databases are the default or go-to option because a table structure is easy to understand and is familiar, but there are many reasons to explore beyond relational databases. E.g. Increased need to process higher volumes, velocities, and varieties of data at a rapid rate has altered the nature of data storage needs for application developers.

NoSQL is a category of databases distinctly different from SQL databases. NoSQL is often used to refer to data management systems that are “Not SQL” or an approach to data management that includes “Not only SQL". There are a number of technologies in the NoSQL category, including document databases, key value stores, column family stores, and graph databases, which are popular with gaming, social, and IoT apps.

When to use NOSQL:

Let's imagine you're building a new social engagement site. Users can create posts and add pictures, videos and music to them. Other users can comment on the posts and give points (likes) to rate the posts. The landing page will have a feed of posts that users can share and interact with.

So how do you store this data? If you're familiar with SQL, you might start drawing something like this:

Here you see to s show the post and the associated images, audio, video, comments, points, and user info on a website or application, you'd have to perform a query with eight table joins just to retrieve the content. Now imagine a stream of posts that dynamically load and appear on the screen and you can easily predict that it's going to require thousands of queries and many joins to complete the task.

In such situation NoSQL option that simplifies the approach for this specific scenario. By using a single document like the following and storing it in DocumentDB, an Azure NoSQL document database service, you can increase performance and retrieve the whole post with one query and no joins. It's a simpler, more straightforward, and more performant result.

{
    "id":"ew12-res2-234e-544f",
    "title":"post title",
    "date":"2016-01-01",
    "body":"this is an awesome post stored on NoSQL",
    "createdBy":User,
    "images":["http://myfirstimage.png","http://mysecondimage.png"],
    "videos":[
        {"url":"http://myfirstvideo.mp4", "title":"The first video"},
        {"url":"http://mysecondvideo.mp4", "title":"The second video"}
    ],
    "audios":[
        {"url":"http://myfirstaudio.mp3", "title":"The first audio"},
        {"url":"http://mysecondaudio.mp3", "title":"The second audio"}
    ]
}

In addition, this data can be partitioned by post id allowing the data to scale out naturally and take advantage of NoSQL scale characteristics. Also NoSQL systems allow developers to loosen consistency and offer highly available apps with low-latency. Finally, this solution does not require developers to define, manage and maintain schema in the data tier allowing for rapid iteration.

Note: In case you still advocate SQL, you could use a relational solution like SQL Server to store the data and query it using joins, as SQL supports dynamic data formatted as JSON.

The following table compares the main differences between NoSQL and SQL.

Hope this helps.

Few links: http://highscalability.com/blog/2010/12/21/sql-nosql-yes.html

Arun Manglick

Microservices (& SOA)

Service Oriented Architecture:

- Check this http://martinfowler.com/bliki/ServiceOrientedAmbiguity.html

- A way of structuring many related applications to work together, rather than trying to solve all problems in one application.

- SOA means so many different things to different people

    - For some SOA is about exposing software through web services and  expect the various WS-* standards and those that will accept any form of XML over http

    - For some SOA implies an architecture where applications disappear.

      Instead core services  supply business functionality and data separated by UI aggregators - That apply presentations that aggregate together the stuff that core services provide.

     - For some SOA is about allowing systems to communicate over some form of standard structure (usually XML based) with other applications - like CORBA

     - For some SOA is all about using (mostly) asynchronous messaging to transfer documents between different systems.

     Essentially this is EAI without all the expensive EAI (Enterprise Application Integration))vendors locking you in.                 

  - ESB (Enterprise Service Bus)

                - Actually a communication system between mutually interacting software applications in a service-oriented architecture (SOA).

     - https://en.wikipedia.org/wiki/Enterprise_service_bus

     - Primary duties of an ESB are:

          - Monitor and control routing of message exchange between services

          - Resolve contention between communicating service components

          - Control deployment and Versioning of services

          - Marshal use of redundant services

          - Cater for commodity services like

                    - Event handling,

                    - Data transformation and mapping,

                    - Message and event queuing and sequencing,

                    - Security or exception handling,

                    - Protocol conversion and

                    - Enforcing proper quality of communication service               

Understanding Microservice Architecture:

In short,

 - microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own 

   process and communicating with lightweight mechanisms, often an HTTP resource API. 

 - These services are built around business capabilities and independently deployable by fully automated deployment machinery. 

 - There is a bare minimum of centralised management of these services, which may be written in different programming languages and use different data storage technologies.

In Detail:

First,

- Software built as microservices can, by definition, be broken down into multiple component services.

-   So that each of these services can be deployed, tweaked, and then redeployed independently without compromising the integrity of an application.

- Its downsides, includes expensive remote calls (instead of in-process calls), coarser-grained remote APIs, and increased complexity when redistributing responsibilities between components.

Second,

- Microservices style is usually organized around business capabilities and priorities and not project

- Unlike a traditional monolithic development approach — where different teams each have a specific focus on, say, UIs, databases, technology layers, or server-side logic— Microservice architecture utilizes cross-functional teams. 

- The responsibilities of each team are to make specific products based on one or more individual services communicating via message bus.

Third,

- Microservices act somewhat like the classical UNIX system: they receive requests, process them, and generate a response accordingly.

- This is opposite to how many other products such as ESBs (Enterprise Service Buses) work, where high-tech systems for message routing, choreography, and applying business rules are utilized.  You could say that microservices have smart endpoints that process info and apply logic, and dumb pipes through which the info flows

Fourth,

- Decentralized governance is favored since involvement of a variety of technologies and platforms, old-school centralized governance aren’t optimal. 

- Microservice encourage developers to save time by always using code libraries established by others, while also giving them the freedom to flirt with alternative solutions when needed.

- Monolithic systems use a single logical database across different applications.  In a microservice application, each service usually manages its unique database

Fifth,

- Like a well-rounded child, microservices are designed to cope with failure.

- Since several unique and diverse services are communicating together, it’s quite possible that a service could fail, for one reason or another.  In these instances, the client should allow its neighbouring services to function while it bows out in as graceful a manner as possible

Finally,

- Microservices architecture is an evolutionary design and, again, is ideal for evolutionary systems where you can’t fully anticipate the types of devices that may one day be accessing your application.

 To sum up: Microservice architecture

  - Uses services to componentize and is usually organized around business capabilities;

  - Focuses on products instead of projects;

  - Has smart end points but not-so-smart info flow mechanisms;

  - Uses decentralized governance as well as decentralized data management;

  - Designed to accommodate service interruptions;

  - Last but not least, is an evolutionary model

Pros:

- Easily be broken down into multiple component services - I.E. Deployed, tweaked, and then redeployed independently 
- Easy integration and automatic deployment (using open-source continuous integration tools such as Jenkins, Hudson, etc.)
- Usually organized around business capabilities and priorities and not project 
- Better fault isolation: if one micro-service fails, the other will continue to work
- Can be developed by a fairly small team
- Code for different services can be written in different languages 

- Easy to scale and integrate with third-party services

Cons:

 - Partitioning the application into micro-services is very much an art
 - Additional effort into implementing communication between the services (ESB)
 - Integration and managing whole products can become complicated
 - Handling use cases that span more than one service is tough
 - Due to distributed deployment, testing can become complicated 

 - Additional complexity of a distributed system

SOA vs. Microservices

-          Some advocates reject the SOA tag altogether.

-          While others consider microservices to be simply an ideal, refined/special form of SOA.

   Note: SOA Tenents

-          Boundaries are explicit

-          Services are autonomous

-          Services share schema and contract, not class

-          Service compatibility is based on policy

               

- Microservices fulfill at least the first two tenets (with a real emphasis on the second), but it's questionable whether they fulfil the third & fourth.

   The reason the third tenet may not hold for microservices is that one of the characteristics of microservices is that they are generally exposed over a RESTful API, which, one could argue, does not expose contract and schema at all (over and above the regular HTTP verbiage).

- Few differences:

-          SOA usually has more dependent ESBs, with microservices using faster messaging mechanisms. 

-          SOA also focuses on imperative programming, whereas microservices architecture focuses on a responsive-actor programming style.

-          SOA models tend to have an outsized relational database, while microservices frequently use NoSQL or micro-SQL databases

-          Microservices is opposite to how many other products such as ESBs (Enterprise Service Buses) work, where high-tech systems for message routing,  choreography, and applying business rules are utilized.  You could say that microservices have smart endpoints that process info and apply logic, and dumb pipes through which the info flows

Well still the real difference has to do with the architecture methods used to arrive at an integrated set of services in the first place.

Hope this helps.

Reference: https://smartbear.com/learn/api-design/what-are-microservices/

Arun Manglick