Introduction to Cloud Computing
This course talks about these topics…
This course talks about these topics…
A model for enabling convenient, on-demand network access to a shared pool of configurable computing resources with minimal management effort or service provider interaction.
Examples of computing resources include:
Three layers in a computing stack:
In the 1950s:
Large-scale mainframes with high-volume processing power.
The practice of time-sharing, or resource pooling, evolved.
Multiple users were able to access the same data storage layer and CPU power.
In the 1970s:
Virtual Machine (VM)
Mainframes to have multiple virtual systems, or virtual machines, on a single physical node
It is no longer a thing of the future
Single individual to Global multi-billion dollar enterprise, anybody can access the computing capacity they need on the cloud.
Cloud makes it possible for businesses to:
Experiment
Fail
Learn Faster than ever before with low risk.
Businesses today have greater freedom to change course than to live with the consequences of expensive decisions taken in the past.
To remain, competitive, businesses need to be able to respond quickly to marketplace changes.
Product lifecycles have shortened, and barriers to entry have become lower.
The power, scalability, flexibility, and pay-as-you-go economics of cloud has made it underpinning foundation for digital transformation.
How can analytics technology leverage the cloud?
It is a form of cloud computing that delivers fundamentals:
compute
network
storage to consumers on-demand, over the internet, on a pay-as-you-go basis.
The cloud provider hosts the infrastructure components traditionally present in an on-premises data center, as well as the virtualization or hypervisor layer.
The ability to track and monitor the performance and usage of their cloud services and manage disaster recovery.
End users don’t interact directly with the physical infrastructure, but experience it as a service provided to them.
Comes with supporting services like auto-scaling and load balancing that provide scalability and high performance.
Object storage is the most common mode of storage in the cloud, given that it is highly distributed and resilient.
A cloud computing model that provides a complete application platform to:
Installation, configuration, operation of application infrastructure:
Servers
Networks
Storage
Operating system
Application runtimes
APIs
Middleware
Databases
User manages: Application Code
A cloud offering that provides access to a service provider’s cloud-based software.
Provider maintains:
Servers
Databases
Application Code
Security
Providers manages application:
Security
Availability
Performance
Organizations are moving to SaaS to:
Reduce on-premise IT infrastructure and capital expenditure
Avoid ongoing upgrades, maintenance, and patching
Run applications with minimal input
Manage websites, marketing, sales, and operations
Gain resilience and business continuity of the cloud provider
Trending towards SaaS integration platforms.
“Cloud infrastructure provisioned for exclusive use by a single organization comprising multiple consumers, such as the business units within the organization. It may be owned, managed, and operated by the organization, a third party, or some combination of them, and it may exist on or off premises.”
An external cloud that offers a private, secure, computing environment in a shared public cloud.
Connects an organization on-premise private cloud and third-party public cloud.
It gives them:
Flexibility
Workloads move freely
Choice of security and regulation features
With proper integration and orchestration between the public and private clouds, you can leverage both clouds for the same workload. For example, you can leverage additional public cloud capacity to accommodate a spike in demand for a private cloud application also known as “cloud bursting”.
After choosing the cloud service model and the cloud type offered by vendors, customers need to plan the infrastructure architecture. The infrastructure layer is the foundation of the cloud.
It is a geographic area or location where a cloud provider’s infrastructure is clustered, and may have names like NA South or US East.
Multiple Availability Zones (AZ)
Have their own power, cooling, networking resources
Isolation of zones improves the cloud’s fault tolerance, decrease latency, and more
very high bandwidth connectivity with other AZs, Data Centers and the internet
Cloud providers offer several compute options:
Virtual servers come with their default local storage, but the stored documents are lost as we destroy the servers. Other more persistent options are:
Networking infrastructure in a cloud datacenter include traditional networking hardware like:
routers
switches
For users of the Cloud, the Cloud providers have Software Defined Networking (SDN), which allows for easier networking:
provisioning
configuration
management
Networking interfaces in the cloud need:
IP address
Subnets
It is even more important to configure which network traffic and users can access your resources:
Security Groups
ACLs
VLANs
VPCs
VPNs
Some traditional hardware appliances:
firewalls
load balancers
gateways
traffic analyzers
Another networking capability provided by the Cloud Providers is:
CDNs
The Cloud provider can choose to de-provision them at any time and reclaim the resources
These VMs are great for:
Non-production
Testing and developing applications
Running stateless workloads, testing scalability
Running big data and HPC workloads at a low cost
Reserve capacity and guarantee resources for future deployments
If you exceed your reserved capacity, complement it with hourly or monthly VMs Note: Not all predefined VMs families or configuration may be available as reserved.
A bare metal server is a single-tenant, dedicated physical server. In other words, it’s dedicated to a single customer.
Processors
RAM
Hard drives
Specialized components
The OS
Add GPUs:
Fully customizable/ demanding environments
Dedicated or long-term usage
High Performance Computing
Highly secure / isolated environments
Bare Metal | Virtual Servers |
---|---|
Work best for: CPU and I/O intensive workloads | Rapidly provisioned |
Excel with the highest performance and security | |
Satisfy strict compliance requirements | Provide an elastic and scalable environment |
Offer complete flexibility, control, and transparency | |
Come with added management and operational over head | Low cost to use |
To create a network in cloud:
It entails creating a set of logical constructs that deliver networking functionality akin to data center networks for securing environments and ensuring high performing business applications.
Containers are an executable unit of software in which application code is packaged, along with its libraries and dependencies, in common ways so that it can be run anywhere—desktops, traditional IT, or the cloud. Containers are lighter weight and consume fewer resources than Virtual Machines.
Direct Attached/Local Storage
File Storage
Block Storage
Object Storage
Disadvantages
Advantages
Like Direct attached:
Attached to a compute node to store data
Unlike Direct attached:
Less expensive
More resilient to failure
Less disk management and maintenance for user
Provision much larger amounts of Storage
Resilient to failure
Offer Encryption
Managed by service provider
File storage is mounted on compute nodes via Ethernet networks:
Departmental file share
‘Landing zone’ for incoming files
Repository of files
i.e., speed variance is not an issue
Input/Output Operations Per Second – the speed at which disks can write and read data.
Block storage breaks files into chunks (or block) of data.
Stores each block separately under a unique address.
Must be attached to a compute node before it can be utilized.
Advantages:
Mounted from remote storage appliances
Extremely resilient to failure
Data is more secure
Mounted as a volume to compute nodes using a dedicated network of optical fibers:
Signals move at the speed of light
Higher price-point
Perfect for workloads that need low-latency
Consistent high speed
Databases and mail servers
Not suitable for shared storage between multiple servers
For block storage, as it is for file storage, you need to take the IOPS capacity of the storage into account:
File Storage | Block Storage |
---|---|
Attached via Ethernet network | Attached via high-speed fiber network |
Speeds vary, based on load | Only attach to one node at a time |
Can attach to multiple computer nodes at once | |
Good for file share where: | |
1) Fast connectivity isn’t required | Good for applications that need: |
2) Cost is a factor | 1) Consistent fast access to disk |
Remember: Consider workload IOPS requirements for both storage types.
When to use Object Storage:
Text files
Audio files
Video files
IoT Data
VM images
Backup files
Data Archives
Not suitable for operating systems, databases, changing content.
Object Storage is priced per/GB
Other costs related to retrieval of the data e.g., Higher access costs for cold vault tiers
Ensure data is stored in correct tier based on frequency of access.
A computing environment that connects an organization’s on-premise private cloud and third-party public cloud into a single infrastructure for running the organization’s applications.
Microservices architecture:
Offloads responsibility for common infrastructure management tasks such as:
Attributes that distinguish serverless computing from other compute models:
Evaluate application characteristics
Ensure that the application is aligned to serverless architecture patterns
Applications that qualify for a serverless architecture include:
Short-running stateless functions
Seasonal workloads
Production volumetric data
Event-based processing
Stateless microservices
Serverless architecture are well-suited for use cases around:
Data and event processing
IoT
Microservices
Mobile backends
Serverless is well-suited to working with:
Text
Audio
Image
Video
Tasks:
Data enrichment
Transformation
Validation and cleansing
PDF processing
Audio normalization
Thumbnail generation
Video transcoding
Data search and processing
Genome processing
Data Streams:
Business
IoT sensor data
Log data
Financial market data
Vendor Dependent Capabilities
Whether creating a new cloud native application or modernizing an existing application:
Dev Teams:
Design Software
Develop Software
Deliver Software
Run Software
Ops Teams
Monitoring
Predicting Failure
Managing Environment
Fixing Issues
A collaborative approach that allows multiple stakeholders to collaborate:
Business owners
Development
Operations
Quality assurance
It applies agile and lean thinking principles to all stakeholders in an organization who develop, operate, or benefit from the business’s software systems, including customers, suppliers, partners. By extending lean principles across the software supply chain, DevOps capabilities improve productivity through accelerated customer feedback cycles, unified measurements and collaboration across an enterprise, and reduced overhead, duplication, and rework.
Using the DevOps approach:
Continuous Delivery
Continuous Integration
Continuous Deployment
Continuous Monitoring
Delivery Pipeline
With its near limitless compute power and available data and application services, cloud computing platforms come with their own risks and challenges, which can be overcome by DevOps:
Tools
Practices
Processes
DevOps provides the following solutions to cloud’s complexities:
Automated provisioning and installation
Continuous integration and deployment pipelines
Define how people work together and collaborate
Test in low-cost, production-like environments
Recover from disasters by rebuilding systems quickly and reliably
Architecture: Monoliths > SOA (Service Oriented Architecture) > Microservices
Infrastructure: Physical servers > VM > Cloud
Delivery: Waterfall > Agile > DevOps
The security in context of cloud is a shared responsibility of:
User
Cloud Provider
Protect data
Manage access
SEC DevOps
Secure Design
Biggest cloud security concerns are:
Data Loss and Leakage
Unauthorized Access
Insecure Interfaces and APIs
Identity and Access Management are:
First line of defense
Authenticate and authorize users
Provide user-specific access
A comprehensive security strategy needs to encompass the security needs of a wide audience:
Organizational users
Internet and social-based users
Third-party business partner organizations
Vendors
There are three main type of users:
Administrative users
Developer users
Application users
Administrators | Operators | Mangers roles that typically create, update, and delete application and instances, and also need insight into their team members’ activities.
An attacker with administrative access could:
Application developers | Platform developers | Application publishers
Can:
Users of the cloud-hosted applications
It is used to combat identity theft by adding another level of authentication for application users.
They are used to securely manage user profiles and their associated credentials and password policy inside a cloud environment.
It helps provide a user-centric view of access to resources or a resource-centric view of access by users:
which users can access which resources
changes in user access rights
access methods used by each user
Critical service within identity and access management framework, both for cloud provider, and cloud consumer.
It enables cloud application/service owners to provision and de-provision:
Streamline access control based on:
Some of the controls that can help secure these sensitive accounts include:
A group of users and service IDs created so that the same access can be assigned to all entities within the group with one or more access policies.
Access policies define how users, service IDs, and access groups in the account are given permission to access account resources.
It plays a key role on cloud, and is often referred to as the last line of defense, in a layered security model.
Scrambling data in a way that makes it illegible.
Encryption Algorithm: Defines rules by which data will be transformed
Decryption Key: Defines how encrypted data will be transformed back to legible data.
It makes sure:
Encryption at Rest:
Block and file storage
Built-in for object storage
Database encryption
Encryption in Transit:
Secure Socket Layer (SSL)
Transport Layer Security (TSL)
Encryption in Use:
Cloud storage encryption could be server-side or client-side.
Server-side:
Create and manage your own encryption keys, or
Generate and manage keys on cloud
Client-side:
Occurs before data is sent to cloud
Cloud providers cannot decrypt hosted data
There is a need to implement a singular data protection strategy across an enterprise’s on-premise, hybrid, and multi-cloud deployments.
Features:
Data access management
Integrated key management
Sophisticated encryption
Multi-cloud encryption console:
Define and manage access policies
Create, rotate, and manage keys
Aggregate access logs
Encryption doesn’t eliminate security risk.
They enable customers to:
Monitoring performance across an entire stack of applications and services can be time-consuming and draining on internal resources.
Cloud monitoring includes:
Strategies
Practices
Processes
Used for:
Analyzing
Tracking
Managing services and apps
It also serves to provide actionable insights that can help improve availability and user experience.
Infrastructure
Help identify minor and large-scale failures
Database
Help track processes, queries, and availability of services
To ensure accuracy and reliability
Application Performance and Monitoring
Help improve user experience
Meet app and user SLAs
To get the most benefit from your cloud-based deployments, you can follow some standard cloud monitoring best practices.
Cloud monitoring needs to be a priority for organizations looking to leverage the benefits of cloud technologies.