Skip to main content

Posts

Showing posts from 2017

Azure Data Lake and Replication mechanism cross Azure Regions

Context
Let’s imagine that we are working in an automotive company that collects telemetric data from their cars. A part of this data needs to be processed, stored and managed.
To be able to store data now and use it later on in time, you decided to go with Azure Data Lake, that is not limited on how much data you can store and allows you to plug any kind of processing system.
Requirements
After the architecture audit, because of legal constraints you are required to have a resiliency policy for disaster recovery. Even if in the same Azure Region there are 3 copies of data that are generated by Azure Data Lake, the legal constraints require you to have a resiliency policy.
Problem
Azure Data Lake makes 3 copies of data in the same Azure Region, but there is no support to replicated or backup content in a different Azure Region. You will need to define you own mechanism for this.

Available Solutions
We could fine many ways of doing this. There are 2-3 mechanism to do replication of Azure …

[Post-Event Event] Event Sourcing and CQRS | ITCamp Community Summer Lunch Event in Cluj

Today we had the first ITCamp Community Event during the lunch break. We decided to do this event at this time in day because it was the only available slot for our special guest Andrea Saltarello.
The talk was about CQRS and Event Sourcing and even if it was only for one hour, the session contained a lot of takeaways, not only from technical perspective, but also from costs and architecture point of view. A great comparison between different NoSQL and ESB systems was presented from Event Sourcing point of view.

There were almost 30 people that decided to transform their lunch to a geek lunch together with ITCamp Community. This event was possible with the support or our local sponsors, that made this event possible.


Below you can find pictures from the event. See you next time!




Azure Audit Logs and Retention Policies

Scope In today post we will talk about Azure Audit Logs and retention policies. Because retention policies might differ from one industry to another, different approaches are required.
Audit Logs From my past experience, I know that each company and department might understand a different thing when you say Audit Logs. I was involved in projects where when you tag a log as audit you would be required by law to keep the audit log for 20-25 years. In this context, I think that the first step for us is to define what is an Audit Log in Azure. In Azure, most of the audit logs can be an activity log or a deployment operation. The first one is close related to any write operation that happened on your Azure Resource (post, put, delete). Read operations are not considered activity logs – but don’t be disappointed, there are many Azure Services that provided monitoring mechanism for read operation also (for example Azure Storage). The second type of audits are the one generated during a dep…

[Community Event] Event Sourcing and CQRS | ITCamp Community Summer Lunch Event in Cluj

At the end of this month (July 24) we will have a special guest in Cluj-Napoca: Andrea Saltarello. The format of the event will be different from the previous ones. The event will take place during the lunch break at The Office and is free.
If you want to find more about the event you can check the following registration links. See you at the event.

Meetup: https://www.meetup.com/ITCamp-Community/events/241394189/
Eventbrite: https://www.eventbrite.com/e/event-sourcing-and-cqrs-itcamp-community-summer-lunch-event-in-cluj-tickets-35994003032
ITCamp Community blog: https://community.itcamp.ro/2017/07/itcamp-community-summer-lunch-event-cluj-event-sourcing-cqrs/

Official announcement:
Let's try a different kind of event this summer. I proposed to all of you to meet during the lunch break and have a talk about Event Sourcing and CQRS. There will be a special guest (Andrea Saltarello - Solution Architect at Managed Design) that will talk about his own experience on how we should manage …

Near real-time analytics for IoT Technician from the field - Azure Time Series Insights

Take a look around you and tell me if you see at least one smart devices capable to send data. There are big chances that you'll have around you more than one. In this moment I have around me a laptop, a Surface, my Fitbit, a SmartTV and a Raspberry PI that fully equipped with weather sensors.
You might say who cares about the data that are collected from them. Maybe nobody or just adds companies. If you would be on a production lines things would be different, you would like to be able to visualize this data from different perspective, analyze them and find way the production fluctuated in a specific day.

Timebound
Data that is collected from different sensors and devices can contain a lot of parameters like temperature, humidity, light and noise level. But in the end when we want to visualize this data, the time information will be the one that will be used on a chart to look at data.
Try to imagine a chart where you put only temperature and humidity information, excluding the ti…

TOGAF® 9 Certification - Architecture Resources for exam preparation

In the last 3 weeks I wasn't active anymore on my blog. This happened because I decided to certified myself as TOGAF 9.

What is TOGAF?
TOGAF is a an architecture framework (Open Group Architecture Framework) for enterprise architectures. The framework comes with support for designing, planning, implement, governance and support an enterprise information technology architecture.
The core of this framework is TOGAF ADM (Architecture Development Method) that describes the method for developing and managing the full lifecycle of an enterprise architecture.

Why TOGAF?
On the market we can find a lot of certificates and standards related to this subject. I decided to go with TOGAF because is one of the frameworks that stays on the foundation of any company when you talk about enterprise architecture.
Additional to this, it is well used in bank, healthcare and life science industry. In comparison with other certificates, you cannot take this exam from your own laptop. It is requested to go…

Part 2 - Overengineering of a cloud application

In the last post we looked over a cloud solution design to ingest small CSV files uploaded by users. This files were crunched by the system that would generate static reports based on the content. Nothing fancy or complex.
The NFR requirements are light, because the real business value stays in the generated reports:

Under 200 users worldwide Concurrency level is 10% (20 users online simultan) Less than 15 CSV uploaded in total per day Basic reporting functionality Current DB size 150MB (2M reporting entries) DB forecast for next 3 years is 1GB (20-25M reporting entries) CSV has up to 1000 entries (maximum 10 columns) The system that was design for this application was a state of the art system - salable, robust, containing all the current technology trends. But of course was over engineering, to powerful and to expensive. Now, the biggest concern was how we can reduce the running cost of the system with a minimal impact (development cost). One of the drivers was that we had to come up…

Part 1 - Overengineering of a cloud application

N-tier architecture is seen nowadays as an old. People tends to migrate to event-base or microservice architecture. It's very common to see people that decide an architecture based on the market trends, ignoring the requirements, business needs and budget.

When you combine this with Azure (cloud in general) you will end up easily with a microservice architecture that combines messaging systems and event-driven architectures. Of course, N-tier application has a lot of disadvantages, but when you have a simple web application, there is no sense to create a complex unicorn that will survive for 100 years.

I was shocked to review a solution that was deployed in Azure 2 months ago, that from architures point of view was beautiful, but from the running and development costs become a nightmare.
I will not go in details of the business requirements, but imagine a system that needs to display some static information, allow users to upload small CSV files that are consolidated in a reportin…

[Post Event] ITCamp Conference 2017 - Cluj-Napoca

What a week! ITCamp Conference took place in Cluj-Napoca Speakers all around the globe joined forces with ITCamp Community team and delivered high quality sessions. As in the last years, topics that are covered by ITCamp Conference were from all technologies - JavaScript to Containers, Azure to Raspberry, OOP to Machine Learning.
The speaker list is pretty long and I invite you to check it out. ITCamp Conference had Google employee's, Principal Program Managers from Microsoft and of course a lot of architects and deep technical people from the field.
Being part of such an event it is a delight. Having live high quality sessions in Cluj-Napoca it is a unique opportunity, offered by ITCamp Community each year.
In figures, ITCamp Community looks very interesting - more than 40 speakers, that deliver 40+ sessions during the two days of the conference to more than 500 attendees.

What a great conference! What a week! Great sessions, great speakers, wonderful people - all of them in one…

[Past Event] DevTalks Cluj-Napoca 2017

This week I was invited at DevTalks to talk about cloud infrastructure and how we can isolated a cloud network from public internet.
DevTalks, as a conference is at the 3rd edition. This year there were 6 track in parallel covering the megatrends of 2017.  It was a good conference, with great speakers and interesting sessions.
Below you can find content related to my session.

Title:
Network isolated inside a cloud environment
Abstract: 
It is possible to create a private network inside a cloud environment that is fully isolated from the external world? If you want to find out the response to this question that you should join the session.
Additional to this we will talk about how we can migrate existing infrastructure to cloud (partially or fully) persisting the same security level as you had before.
Slides:

Network isolated inside a cloud environment Radu Vunvulea DevTalks 2017 Cluj Romania from Radu Vunvulea
Pictures:



Azure Cosmos DB | The perfect place for device topology for world wide solutions

In the world of IoT, devices are distributed all around the world. The current systems that are now on the market are offering scalable and distributed systems for communication between devices and our backends.

Context
It is not something out of ordinary if we have an IoT solution distributed to 3 or 5 places around the globe. But behind this performant systems we need a storage solution flexible enough for different schemas but in the same time to be powerful enough to scale at whatever size we want.

Relational database are used often to store data where we have a stable schema. Having different devices across the globe request to have different schemas and data formats. Storing data in non-relational databases is more natural and simple. A key-value, graph or document database is many time more suitable for our needs in IoT scenarios then a relational database.

Current solutions
There are plenty solutions on the market, that are fast, powerful and easy to use. I expect that you heard…

Azure Key Vault | How Secrets and Keys are stored

I'm pretty sure that most of you heard about Azure Key Vault. If not I recommend to take a look over this page that describes in details how Azure Key Vault helps us as a safeguard for our application secrets and cryptographic keys (like certificates).

Scope
The main scope of this post is to take a look on how our secrets are stored. This is important because there are keys that cannot be recovered once generated or stored and we might end up without keys in the case we lose them.

What is HSM?
HSM is an acronym for Hardware Security Module. It is a physical device that can manage digital keys by providing cryptographic capabilities. HSM is playing the role of a safeguard by offering cryptographic capabilities directly by the hardware.

Is the tuple <keys, secrets> stored inside HSM?

No, there is no need to store this information in HSM. Secrets are stored outside the HSM, but they are encrypted using a key chain that terminates inside the HSM.
An analogy related to key chains an…

New Azure role - Azure Billing Reader

There is a small new feature on Azure Billing that made my day great. Small things matters and in this case is 100% applicable.

Problem
Until now there was no mechanism to share ONLY billing information with users. You could forward billing information to specific users. This was useful for last month consumptions, but nothing that you could do directly for the current month.

Current solution
Some shortcuts could be done to make available this information.
The most simple way is  to give to the user the Co-Administrator rights. This will work great, but that user will have access to other resources also, not only to the billing data. Having access to all resources can be a downside, especially if you need to give access to a non-technical person that might click the wrong button (smile). You might reduce the access by using RBAC (Role-Based Access Control), but you would need some extra configuration steps.
Another approach is by exporting billing and cost information using Azure REST …

Control Azure Users Access using Role-Based Access Control

Problem As a customer I want to be able to restrict user access and rights to Azure Resources that are under the same Azure Subscription.
The requirement can be extended a little more by specifying that a user needs to be able to view and access only Azure Resources that he is allowed. He shall be able to create or modify resources only specific resources.    If possible, all resources that user access or create should be under a predefined subnet.
Options There are multiple approaches to a request like this. Even if Role-Based Access Control is a powerful mechanism to control access, the way how we can restrict access is limited and doesn't allow us to restrict fully the user as we need. A classical approach is to allow user to run ARM (Azure Resource Management) scripts only through a custom component that is hosted by us. Having full control on this component we can implement any kind of logic and business restriction. The downside of a solution like this is complexity and cost.…

[Post Event] Global Azure Bootcamp event, Cluj-Napoca, April 22th, 2017

For the 5th year in the row,  ITCamp community from Cluj-Napoca joined Global Azure Bootcamp event. In comparison with other communities events organized by in Cluj-Napoca, Global Azure Bootcamp is different because is purely hands-on lab.
There were 3 different workshops of 90 minutes each where attendees had the opportunity to play and test Azure Functions, Machine Learning and Azure Resources Managers (ARM). We respected this year also the tradition to organize the event at Endava (ISDC) building, where the location is perfect for hands-on labs. Special thank you for our local sponsor - Endava.

Agenda
09:00-09:30 - Sosirea participanților
09:30-11:00 - Azure Functions (Radu Vunvulea)
11:00-12:30 - Machine learning for mere mortals with Azure ML (Silviu Niculita)
12:30-14:00 - ARM Templates, how to create them, and use them in your CD pipeline (Florin Loghiade)

Workshops descriptions
Azure Functions (Radu Vunvulea) 

What are Azure Functions? AWS Lambda from Azure. This is the fastest w…

[Post Event] Microsoft Data Amp—where data gets to work meet-up

Together with ITCamp community from Cluj-Napoca we joined our forces for 2 hours. We wanted to find more about how we can transform our business with our data -Microsoft Data Amp.
A lot of new stuff were announced during the live streaming that can be viewed on https://www.microsoft.com/en-us/sql-server/data-amp.
If you ask me, the game changers are SQL Server 2017 that runs on Linux and the fully integration between SQL Server and other stacks like R. Nowadays you don't need to move your data outside the Database engine to execute ML. You have the ML full support direct in the database engine.



Running Azure Stream Analytics on Edge Devices

Azure Stream Analytics enable us to analyze data in real time. Connecting multiple streams of data, running queries on top of them without having to deploy complex infrastructure become possible using Azure Stream Analytics.
This Azure service is extremely powerful when you have data in the cloud. But there was no way to analyze the data stream at device or gateway level.
For example if you are in a medical laboratory you might have 8 or 12 analyzers. To be able to analyze the counters of all this devices to detect a malfunction you would need to push counters value to Azure, even if you don't need for other use cases.

Wait! There is a new service in town that enable us to run the same Azure Stream Analytics queries but at gateway or device level - Azure Stream Analytics on Edge Devices. Even if the name is long and has Azure in the name, the service is a stand alone service that runs on-premises.
This service allows us to run the same queries in real time over data streams that w…

Global Azure Bootcamp la Cluj-Napoca | April 22, 2017

Acesta este al cincilea an când ITCamp community organizează Global Azure Boot Camp. Acesta este un eveniment la nivel global care are loc în peste 159 de locații. Ca și anul trecut, Clujul nu se lasă mai prejos și apare pe harta Azure. Pe data de 22 Aprilie vă invităm pe toți la acest eveniment din Cluj-Napoca, care va conține 3 workshop-uri. Link pentru înregistrare: https://itcamp-cj-2017-azurebootcamp.eventbrite.com Participarea la eveniment este GRATUITĂ, așa cum a fost și până acum la orice eveniment organizat de comunitatea ITCamp. Pe data de 22 Aprilie ne propunem să avem 3 workshop-uri de câte 90 de minute fiecare, unde putem să învățăm împreună cum să folosim diferite servicii Azure. Fiecare workshop conține o parte teoretică și una practică. Din această cauză, o să aveți nevoie de un laptop. De ce aveți nevoie: Laptop + Visual Studio + Microsoft Azure SDK + Un cont de Azure Dacă doriți puteți să vă grupați în grupuri de 2-3 persoane la același laptop. Link pentru înregistra…