Why NoSQL (vs SQL)

NoSQL databases that enable storing unstructured and heterogeneous data at scale have gained in popularity. For most developers, relational databases are the default or go-to option because a table structure is easy to understand and is familiar, but there are many reasons to explore beyond relational databases. E.g. Increased need to process higher volumes, velocities, and varieties of data at a rapid rate has altered the nature of data storage needs for application developers.

NoSQL is a category of databases distinctly different from SQL databases. NoSQL is often used to refer to data management systems that are “Not SQL” or an approach to data management that includes “Not only SQL". There are a number of technologies in the NoSQL category, including document databases, key value stores, column family stores, and graph databases, which are popular with gaming, social, and IoT apps.

When to use NOSQL:

Let's imagine you're building a new social engagement site. Users can create posts and add pictures, videos and music to them. Other users can comment on the posts and give points (likes) to rate the posts. The landing page will have a feed of posts that users can share and interact with.

So how do you store this data? If you're familiar with SQL, you might start drawing something like this:

Here you see to s show the post and the associated images, audio, video, comments, points, and user info on a website or application, you'd have to perform a query with eight table joins just to retrieve the content. Now imagine a stream of posts that dynamically load and appear on the screen and you can easily predict that it's going to require thousands of queries and many joins to complete the task.

In such situation NoSQL option that simplifies the approach for this specific scenario. By using a single document like the following and storing it in DocumentDB, an Azure NoSQL document database service, you can increase performance and retrieve the whole post with one query and no joins. It's a simpler, more straightforward, and more performant result.

{
    "id":"ew12-res2-234e-544f",
    "title":"post title",
    "date":"2016-01-01",
    "body":"this is an awesome post stored on NoSQL",
    "createdBy":User,
    "images":["http://myfirstimage.png","http://mysecondimage.png"],
    "videos":[
        {"url":"http://myfirstvideo.mp4", "title":"The first video"},
        {"url":"http://mysecondvideo.mp4", "title":"The second video"}
    ],
    "audios":[
        {"url":"http://myfirstaudio.mp3", "title":"The first audio"},
        {"url":"http://mysecondaudio.mp3", "title":"The second audio"}
    ]
}

In addition, this data can be partitioned by post id allowing the data to scale out naturally and take advantage of NoSQL scale characteristics. Also NoSQL systems allow developers to loosen consistency and offer highly available apps with low-latency. Finally, this solution does not require developers to define, manage and maintain schema in the data tier allowing for rapid iteration.

Note: In case you still advocate SQL, you could use a relational solution like SQL Server to store the data and query it using joins, as SQL supports dynamic data formatted as JSON.

The following table compares the main differences between NoSQL and SQL.

Hope this helps.

Few links: http://highscalability.com/blog/2010/12/21/sql-nosql-yes.html

Arun Manglick

Microservices (& SOA)

Service Oriented Architecture:

- Check this http://martinfowler.com/bliki/ServiceOrientedAmbiguity.html

- A way of structuring many related applications to work together, rather than trying to solve all problems in one application.

- SOA means so many different things to different people

    - For some SOA is about exposing software through web services and  expect the various WS-* standards and those that will accept any form of XML over http

    - For some SOA implies an architecture where applications disappear.

      Instead core services  supply business functionality and data separated by UI aggregators - That apply presentations that aggregate together the stuff that core services provide.

     - For some SOA is about allowing systems to communicate over some form of standard structure (usually XML based) with other applications - like CORBA

     - For some SOA is all about using (mostly) asynchronous messaging to transfer documents between different systems.

     Essentially this is EAI without all the expensive EAI (Enterprise Application Integration))vendors locking you in.                 

  - ESB (Enterprise Service Bus)

                - Actually a communication system between mutually interacting software applications in a service-oriented architecture (SOA).

     - https://en.wikipedia.org/wiki/Enterprise_service_bus

     - Primary duties of an ESB are:

          - Monitor and control routing of message exchange between services

          - Resolve contention between communicating service components

          - Control deployment and Versioning of services

          - Marshal use of redundant services

          - Cater for commodity services like

                    - Event handling,

                    - Data transformation and mapping,

                    - Message and event queuing and sequencing,

                    - Security or exception handling,

                    - Protocol conversion and

                    - Enforcing proper quality of communication service               

Understanding Microservice Architecture:

In short,

 - microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own 

   process and communicating with lightweight mechanisms, often an HTTP resource API. 

 - These services are built around business capabilities and independently deployable by fully automated deployment machinery. 

 - There is a bare minimum of centralised management of these services, which may be written in different programming languages and use different data storage technologies.

In Detail:

First,

- Software built as microservices can, by definition, be broken down into multiple component services.

-   So that each of these services can be deployed, tweaked, and then redeployed independently without compromising the integrity of an application.

- Its downsides, includes expensive remote calls (instead of in-process calls), coarser-grained remote APIs, and increased complexity when redistributing responsibilities between components.

Second,

- Microservices style is usually organized around business capabilities and priorities and not project

- Unlike a traditional monolithic development approach — where different teams each have a specific focus on, say, UIs, databases, technology layers, or server-side logic— Microservice architecture utilizes cross-functional teams. 

- The responsibilities of each team are to make specific products based on one or more individual services communicating via message bus.

Third,

- Microservices act somewhat like the classical UNIX system: they receive requests, process them, and generate a response accordingly.

- This is opposite to how many other products such as ESBs (Enterprise Service Buses) work, where high-tech systems for message routing, choreography, and applying business rules are utilized.  You could say that microservices have smart endpoints that process info and apply logic, and dumb pipes through which the info flows

Fourth,

- Decentralized governance is favored since involvement of a variety of technologies and platforms, old-school centralized governance aren’t optimal. 

- Microservice encourage developers to save time by always using code libraries established by others, while also giving them the freedom to flirt with alternative solutions when needed.

- Monolithic systems use a single logical database across different applications.  In a microservice application, each service usually manages its unique database

Fifth,

- Like a well-rounded child, microservices are designed to cope with failure.

- Since several unique and diverse services are communicating together, it’s quite possible that a service could fail, for one reason or another.  In these instances, the client should allow its neighbouring services to function while it bows out in as graceful a manner as possible

Finally,

- Microservices architecture is an evolutionary design and, again, is ideal for evolutionary systems where you can’t fully anticipate the types of devices that may one day be accessing your application.

 To sum up: Microservice architecture

  - Uses services to componentize and is usually organized around business capabilities;

  - Focuses on products instead of projects;

  - Has smart end points but not-so-smart info flow mechanisms;

  - Uses decentralized governance as well as decentralized data management;

  - Designed to accommodate service interruptions;

  - Last but not least, is an evolutionary model

Pros:

- Easily be broken down into multiple component services - I.E. Deployed, tweaked, and then redeployed independently 
- Easy integration and automatic deployment (using open-source continuous integration tools such as Jenkins, Hudson, etc.)
- Usually organized around business capabilities and priorities and not project 
- Better fault isolation: if one micro-service fails, the other will continue to work
- Can be developed by a fairly small team
- Code for different services can be written in different languages 

- Easy to scale and integrate with third-party services

Cons:

 - Partitioning the application into micro-services is very much an art
 - Additional effort into implementing communication between the services (ESB)
 - Integration and managing whole products can become complicated
 - Handling use cases that span more than one service is tough
 - Due to distributed deployment, testing can become complicated 

 - Additional complexity of a distributed system

SOA vs. Microservices

-          Some advocates reject the SOA tag altogether.

-          While others consider microservices to be simply an ideal, refined/special form of SOA.

   Note: SOA Tenents

-          Boundaries are explicit

-          Services are autonomous

-          Services share schema and contract, not class

-          Service compatibility is based on policy

               

- Microservices fulfill at least the first two tenets (with a real emphasis on the second), but it's questionable whether they fulfil the third & fourth.

   The reason the third tenet may not hold for microservices is that one of the characteristics of microservices is that they are generally exposed over a RESTful API, which, one could argue, does not expose contract and schema at all (over and above the regular HTTP verbiage).

- Few differences:

-          SOA usually has more dependent ESBs, with microservices using faster messaging mechanisms. 

-          SOA also focuses on imperative programming, whereas microservices architecture focuses on a responsive-actor programming style.

-          SOA models tend to have an outsized relational database, while microservices frequently use NoSQL or micro-SQL databases

-          Microservices is opposite to how many other products such as ESBs (Enterprise Service Buses) work, where high-tech systems for message routing,  choreography, and applying business rules are utilized.  You could say that microservices have smart endpoints that process info and apply logic, and dumb pipes through which the info flows

Well still the real difference has to do with the architecture methods used to arrive at an integrated set of services in the first place.

Hope this helps.

Reference: https://smartbear.com/learn/api-design/what-are-microservices/

Arun Manglick

MVC vs MVP vs MVVM

Read my earlier post - http://arun-ts.blogspot.jp/2010/12/mvvm-silverlight.html

MVC –, 1979

MVP – Model-Model-View-ControllerView-Presenter, 90s

Modern MVVM – Model-View-ViewModel, 2005

General rules for when to use which?

 

MVP

• Use in situations where binding via a datacontext is not possible.
• Windows Forms is a perfect example of this.  
• In order to separate the view from the model, a presenter is needed.  Since the view cannot directly bind to the presenter, information must be passed to it view an interface (IView).

MVVM

• Use in situations where binding via a datacontext is possible.  Why?  The various IView interfaces for each view are removed which means less code to maintain.
• Some examples where MVVM is possible include WPF and javascript projects using Knockout.
• MVVM separates the presentation part and business logic. This leads to: 
• Higher testability
• Higher maintainability

MVC

• Use in situations where the connection between the view and the rest of the program is not always available (and you can't effectively employ MVVM or MVP).
• This clearly describes the situation where a web API is separated from the data sent to the client browsers.  
• Microsoft's ASP.NET MVC is a great tool for managing such situations and provides a very clear MVC framework

Model-View-Presenter

<![if !supportLists]>1.       <![endif]>In MVP, the Presenter contains the UI business logic for the View.

<![if !supportLists]>2.     <![endif]>All invocations from the View delegate directly to Presenter.

<![if !supportLists]>3.     <![endif]>The Presenter is also decoupled directly from the View and talks to it through an interface. This is to allow mocking of the View in a unit test. I.e. The Presenter is not responsible for determining which View is displayed in response to any action including when the application loads, instead specific view instance is passed to the Presenter on the load of the view.

<![if !supportLists]>4.     <![endif]>Here View has a reference to Presenter (But completely decoupled from each other's and communicate to each other's by an interface) but View has no reference to Model.

public partial class Views_add_category_view : System.Web.UI.UserControl, IAddCategoryView

{

  protected void Page_Load(object sender, EventArgs e)

  {

    presenter = new AddCategoryPresenter(this);

  }

}

One common attribute of MVP is that there has to be a lot of two-way dispatching. For example, when someone clicks the "Save" button, the event handler delegates to the Presenter's "OnSave" method. Once the save is completed, the Presenter will then call back the View through its interface so that the View can display that the save has completed.

Imp: MVP tends to be a very natural pattern for achieving separated presentation in Web Forms. The reason is that the View is always created first by the ASP.NET runtime.

Two primary variations of MVP

Passive View: The View is as dumb as possible and contains almost zero logic. The Presenter is a middle man that talks to the View and the Model. The View and Model are completely shielded from one another. +

The Model may raise events, but the Presenter subscribes to them for updating the View.

Similarly View exposes setter properties which the Presenter uses to set the data in the Model.

All state is managed in the Presenter and not the View.

• Pro: maximum testability surface; clean separation of the View and Model
• Con: more work (for example all the setter properties) as you are doing all the data binding yourself.

Supervising Controller: The Presenter handles user gestures. The View binds to the Model directly through data binding.

In this case it's the Presenter's job to pass off the Model to the View so that it can bind to it (Like in ASP.NET MVC). The Presenter will also contain logic for gestures like pressing a button, navigation, etc.

• Pro: by leveraging databinding the amount of code is reduced.
• Con: there's less testable surface (because of data binding), and there's less encapsulation in the View since it talks directly to the Model.

Model-View-Controller

<![if !supportLists]>1.       <![endif]>In MVC, the Controller contains the UI business logic for the View.

<![if !supportLists]>2.    <![endif]>All action in the View correlates with a call to a Controller along with an action.

<![if !supportLists]>3.     <![endif]>In the MVC, Controller is tightly coupled to the View. I.e. Controller is responsible for determining which View is displayed in response to any action including when the application loads. This differs from MVP where actions route through the View to the Presenter. Also in MVC, every action in the View correlates with a call to a Controller along with an action. In the web each action involves a call to a URL on the other side of which there is a Controller who responds. Once that Controller has completed its processing, it will return the correct View. The sequence continues in that manner throughout the life of the application:

    Action in the View

        -> Call to Controller

        -> Controller Logic

        -> Controller returns the View.

One other big difference In implementations of MVC with MVP, the View usually will not have any logic in the code behind.

This is contrary to MVP where it is absolutely necessary as if the View always has delegation code to the Presenter.

Another big difference - Unlike view and controller (In MVC), view and presenter (In MVP) are completely decoupled from each other's and communicate to each other's by an interface

Today, this pattern is used by many popular framework like as Ruby on Rails, Spring Framework, Apple iOS Development and ASP.NET MVC.

Presentation Model / Model-View-ViewModel.

One other pattern to look at is the Presentation Model pattern. In this pattern there is no Presenter. Instead the View binds directly to a Presentation Model. The Presentation Model is a Model crafted specifically for the View. This means this Model can expose properties that one would never put on a domain model as it would be a violation of separation-of-concerns. In this case, the Presentation Model binds to the domain model, and may subscribe to events coming from that Model. The View then subscribes to events coming from the Presentation Model and updates itself accordingly. The Presentation Model can expose commands which the view uses for invoking actions. The advantage of this approach is that you can essentially remove the code-behind altogether as the PM completely encapsulates all of the behaviour for the view. This pattern is a very strong candidate for use in WPF applications and is also called Model-View-ViewModel.

Reference:

http://stackoverflow.com/questions/19444431/what-is-difference-between-mvc-mvp-mvvm-design-pattern-in-terms-of-coding-c-sh

http://joel.inpointform.net/software-development/mvvm-vs-mvp-vs-mvc-the-differences-explained/

Hope this helps.

Thanks & Regards,

Arun Manglick,

01- What Is AppFabric Cache

Windows Server AppFabric provides Distributed Caching features.

 

Windows Server AppFabric Caching features use a cluster of servers that communicate with each other to form a single, unified application cache system.

As a distributed cache system, all cache operations are abstracted to a single point of reference, referred to as the Cache Cluster.

In other words, your client applications can work with a single logical unit of cache in the cluster regardless of how many computers make up the cache cluster.

 

The primary components of the physical architecture consist of:

 

1.       Cache Server

2.       Cache Host Windows Service

3.       Cache Cluster (Cluster of Cache Servers)

4.       Windows Power Shell-Based Cache Administration Tool

5.       Cluster Configuration Storage Location

6.       Cache Client

 

The following diagram shows how all of these elements relate.

 

 

 

Cache Client	Any application server that is running a cache-enabled application may be loosely referred to as the cache client. For an application to be cache-enabled, it must use the AppFabric caching assemblies and specify the appropriate application configuration settings programmatically or in an XML-based application configuration file
Cache Cluster	The cache cluster is a collection of one or more instances of the Caching Service working together in the form of a ring to store and distribute data. Data is stored in memory to minimize response times for data requests. The operations of the cache cluster are managed by a role, named the cluster management role
Cache Hosts	The AppFabric Caching Service is a Windows service that runs on one or more servers.
Cache Server	Each server that runs the Caching Service is referred to as a cache server. For each cache server, only one instance of the Caching Service can be installed.
Windows Power Shell-Based Cache Administration Tool	Windows Power Shell is the exclusive management tool for the Cache Service. Windows PowerShell cache administration cmdlets can be installed on any domain computer or on the cache servers themselves. This is done by installing the Cache Administration feature of AppFabric. You must have Administrator privileges on all of the cache servers for the tool to function properly.
Cluster Configuration Storage Location	Each time the cluster starts, it must retrieve configuration information from the cluster configuration storage location. The generic term "storage location" is used because the location is determined by how you choose to store the cluster configuration settings.

 

Architecture Overview of App Fabric

 

The diagram below provides a high-level overview of the AppFabric system.

 

 

 

·      Apps are deployed into a farm of AppFabric servers that share Workflow Instance  Stores and Monitoring Stores (Databases).

·      Distributed Cache provides unified data cache view to Apps while distributing and replicating data across multiple machines

 

The Windows Server AppFabric Cache stores Dot Net objects, scales seamlessly, and manages data location and redundancy by distributing the cache across a cluster of machines.  

The developer can simply put data in the cache and retrieve it when needed - AppFabric handles all the underlying complexity for you.

 

Functional View

 

The diagram below illustrates key components of the AppFabric system and their functions from a single-machine view (one AppFabric Server view) in a typical setup.

 

 

 

 

Thus Windows Server AppFabric adds improved hosting capabilities to Windows Server to efficiently host all your WCF services, including WF Services.

Additionally, AppFabric provides common services that help you as a developer build scalable Web and composite applications.

 

There are many NFRs which are ultimately getting addressed with App Fabric. Some of them are :

 

·      Reliability – Overall (pertains to system’s maturity, fault tolerance level, recoverability and product relevance duration)

·      Reliability – Availability ( The fraction of the time when the system is operative and accessible to all it users)

·      Reliability – Integrity ( The probability that a system will not be compromised such as to perform incorrectly or return incorrect results)

·      Reliability – Security ( The probability that a system will not be compromised via access for read- write operations by unauthorized users or other software )

·      Monitor ability : ( Ability to monitor run time - sessions, auditing of transactions )

·      Stability ( a facet of performance, availability and manageability; devoid of memory leaks , dangling areas and other creeping issues)

·      Availability –HA ( with Recoverability as a sub-requirement. Also MTBF – the mean time expected between two run time failures  MTTR the mean time required to fix a bug after it is noticed at run time;  software failover, hardware failover, standby hardware and so on ; strict ACID transactions, Tracing and Auditing; Business continuity against disasters; Recovery from malicious attacks )

 

Note:

 

AppFabric Caching requires less management; the "fabric" takes care of everything. 

Adding new machines to the cache is handled in the AppFabric Configuration Wizard. 

The cache takes a central configuration approach, so all the servers in the cache cluster know about each other. 

You can choose to store this configuration in SQLServer or in a file share.   

 

 

Distributed Cache

 

The Distributed Cache of App Fabric provides a distributed, in-memory application cache for developing scalable, available, and high-performance applications.

 

Distributed Cache Service runs across multiple machines and forms a tight cluster offering data replication and data consistency across multiple machines.

Distributed Cache Service can run on the same machine as the application code or run in a remote dedicated farm of machines.

 

 

How To Setup AppFabric Cache

 

Please check the link below.

 

http://www.hanselman.com/blog/InstallingConfiguringAndUsingWindowsServerAppFabricAndTheVelocityMemoryCacheIn10Minutes.aspx

 

 

 

Thanks & Regards,

Arun Manglick,

Project Manager - Forecasting
Mobile: 9158041782| 9850901262| Desk Phone: +912040265348 | http://www.synechron.com

SYNECHRON -
- 4,000+ professionals globally.
- USA | Canada | UK | The Netherlands | UAE | India | Singapore | Hong Kong | Japan

 

02- How to SetUp AppFabric Cache

How To Setup AppFabric Cache

 

1.       Download and Install App Fabric.

2.       Run the installer and select AppFabric Cache. The configuration tool will pop up, and walk you through a small wizard.

 

 

1.    Accept the default settings on Customer Experience page as shown in below image.

2.    On the Features Selection Page select all components for installation.

3.    On the Confirmation Page Click ‘Install’

4.    Click ‘Finish’ on the Results page.

 

5.    This will launch the Configuration Wizard as shown below.

 

 

6.    Select ‘Next’ on the above Welcome Screen above to get the next screen.

7.    Select ‘Next’ on the above Hosting Screen above to get the following screen.

Select all the option as mentioned in below screen.

 

 

 

8.    Then press Configure button and the following window appears.

Select ‘Register AppFabric Caching Service configuration database’ and ‘Create AppFabric Caching Service configuration database’.

 

 

9.    Press OK.

10. Then Select ‘Next’ on the Configure Caching Service Wizard page - Step 9

11. Press Yes on the next dialog box.

12. Then press Next on the Cache Node wizard page (With Default Settings)

 

 

13. Select Yes on the next dialog box.

14. On the last page click Finish the Cache Configuration.

 

 

Reference:

Please check the link below.

http://www.hanselman.com/blog/InstallingConfiguringAndUsingWindowsServerAppFabricAndTheVelocityMemoryCacheIn10Minutes.aspx

 

 

Regards,

Arun Manglick