DAD - Distributed Apps Dev and Cloud - Dezvoltarea Aplicatiilor Distribuite si Cloud

Mission

Technological transfer from university to the students of practical and theoretical issues related with Distributed Applications Development and Cloud.

Please follow the info @:

* For the period September 2023 - February 2024, we have at room 2001 the following activities: Lecture - on Tue from 18:00 | Seminar/Laboratories - on Tue, from 19:30

We have used this platform - Zoom including in the university rooms (old link: https://us02web.zoom.us/j/82880201368?pwd=SitqdW9NUDNMVTd3VnRBSHJucENPUT09), because we had to compensate the poor image quality from the video projector and as preventive "training" for any unforeseen situations. The Cyber Security Master Program: http://www.ism.ase.ro has been upgraded on Tue, March 9-th, 2020, the Zoom platform, so that the working session will be extended from 40 minutes to 12 hours.

The presence is mandatory both in the course, as well as in the seminar / lab.

Lectures & Labs

No.  Lecture 
PDF		 Lecture Description Sources for the 
Lecture		 Lab Description Sources for the
Lab 
-

INTRO - Distributed Applications Development administrative issues

1.

RECAP: Java Multi-threading mechanisms, java.util.concurrent package in samples for HPC - High Performance Computing samples | IPC - Inter-Process Communication in Linux using C/C++, Linux C POSIX Multi-Threading (PThread), Java Multi-threading and Parallel Computing Intro. Multi-tenancy via multi-process or multi-threading.

For the Java intro full details (RECAP sections), please see the lectures and labs of Multi-paradigm Programming - Java @ http://acs.ase.ro/java and first three activities

For Parallel Computing (incl. C OpenMP) intro please see lectures and labs details @ http://acs.ase.ro/parallel-processing

Distributed vs Parallel Computing Systems
* HTC (High Throughput Computing) vs. HPC (High Performance Computing)

 RECAP: Java Multi-threading mechanisms, java.util.concurrent package in samples for HPC - High Performance Computing samples 
2.

RECAP: TCP/IP Network Concepts - OSI Model, IP v4 vs. v6, UDP headers, NAT, PAT, Network Programming in Java - UDP, SNMP client over UDP sample - network monitoring (run in Docker containers (Native Cloud: CaaS - Container as a Service).

ANT, Maven, Gradle, OS VM vs. Docker - virtualization, DevOps Jenkins Intro

Intro in Java Reflection & Annotations, JNI - Java Native Interface, Java Exception mechanism, Source Code Design Patterns - Singleton.

 RECAP: Network UDP - Java UDP Network programming using Socket, SNMP Client sample - simple and with library snmp4j run in Docker containers (Native Cloud: CaaS - Container as a Service)
3.

RECAP: Java 8+ Lamda with Multi-threading, TCP/IP Network Concepts - Network Programming in Java - UDP & TCP with multi-threading, IP, TCP headers, TCP state machine, NAT, PAT, Network Programming in Java - TCP

 RECAP: Java 8+ Lambda - Network HTTP-Multi-threading and TCP with lambda Example - web server 
4.

Java Servlet Intro - Apache Tomcat web server with servlet container distribution, Session Tracking, HTTP Requests and Responses, Cookies, HTTP 1.1 

Cloud PaaS Intro: Deployment of Servlet into Google App Engine - PaaS Cloud.

HTTP/1.1 understanding with Wireshark - Java Servlet Sample with annotation instead web.xml descriptor, deployment in Apache Tomcat 9 + processing a HTML form with Java Servlet.
5.

Java Server Pages - JSP Intro - Apache Tomcat web server with Servlet/JSP container distribution, JSP Architecture, Directives and Actions, Objects, Tag-Libs, Beans, MVC - Model View Controller Type 1 and 2 Intro

Servlet and JSP are part of Web curriculum per se, and they are requred in this disscipline as data gathering/Cloud Gateway for massive distributed and parallel computing in back-end servers: Cloud, GRID, Fog infrastructures 

Java Server Pages - JSP Sample deployment in Apache Tomcat 9 + processing a HTML form with JSP and SQLite/MongoDB access.

Local deployment into VMBox.
6.

RPC - Remote Procedure Call and Java RMI - Remote Method Invocation

 Java RMI sample for adding two integer vectors.
7.

SOA (Service Oriented Architectures) WebServices SOAP - Simple Objects Access Protocol

Process Migrations vs. Data Migrations within a Distributed System | TCP Socket example for passing business logic and data migration

SOA WebServices SOAP - Java JAX-WS in Eclipse (XML Recap)

Stand-alone client and server application

JAX-WS Web Server app

8.

RESTlful Services  vs. SOA | JAX-RS with HTTP/2

Difference between HTTP/1.1 and HTTP/2.0 with Wireshark network packets analyzer

HTTP/2 Client API with JDK11+

Open MPI

Cloud IaaS Intro: Deployment of OpenMPI programs in Amazon EC2

 RESTful service in Java JAX-RS in Eclipse (JSON Recap)
T9.

TEST on PC

 TEST on PC
9.

node.js INTRO/RECAP

Cloud FaaS - Functions as a Service / AWS Lamda Sever-less Programming Example vs. Fog Computing 

Angular.js vs. REACT vs. Apache Cordova + WebAssembly - Web/Mobile Front-End Development is NOT part (beyond) of this curriculum of this lecture - they should be part of Web or Mobile topic

 Node.js JavaScript/ECMAScript - incl. WebServer & RESTful service in Node.js + WebSockets (Java/Node.js)
10.

Middleware & MoM: JMS vs. Apache Kafka

 Middleware: Apache JMS and Kafka implementation for the Producer and Consumer Clients example
11.

Distributed Components: EJB - Enterprise Java Beans / Java | Jakarta EE platform vs. Actors in Akka 

 Apache EJB 3.x and MDB - Message Driven Bean vs. Akka actors examples
12.

Intro n-Tier Architecture, Back-end Servers Clustering & Servers Infrastructures & (Native) Clouds - IaaS, PaaS, CaaS, SaaS, FaaS and Fog Computing

Infrastructures for Microservices / PODs and Containers as a Service Cloud - CaaS - Docker | Kubernetes

 Spring Boot: JMS and components beans plus RESTful Services 

13.

14.

Spring Microservices in Action with Java/Kotlin and WASM - Web Assebmly as alternative to Java Applets technology

 Spring End2End Microservice Example

E-Learning Solution & E-Framework Solutions

If necessary, the E-Learning solution platform is based on Moodle and SAKAI CLEand it is provided by ISM - IT | Cyber Security Master program: http://ism.ase.ro and ACS - Applied Computer Science And Cyber Security Team: http://acs.ase.ro. Moodle and SAKAI e-Learning Systems are available (OLD address: http://37.120.250.22:8020/portal )

E-Framework solutions for Distributed Applications Development and cloud (DAD) lectures and labs are stored in Git. The students may get them into the Oracle VM VirtualBox client virtual machine or Docker Container Image, and they are provided by Cyber Security Master program - http://ism.ase.ro
The details are the following:

1. The source code for the lectures and labs is in Git @: https://github.com/critoma/dad 

2. Docker Container Image with Linux Ubuntu 20.04 LTS for download (docker pull critoma/linux-ubuntu-dev-security-ism)

https://hub.docker.com/r/critoma/linux-ubuntu-dev-security-ism

Evaluation & Gradebook Mark Solution 

The final exam mark is 70%, and the assesments/homeworks/reports/test&quizzes done by the student, during the semester are 30% from the evaluation mark.
All the content related with the tests, assesments and the final exam will be uploaded accordinglly with the established deadlines in the E-Learning System Solution based on Moodle or SAKAI and provided by IT&C Security Master program - http://ism.ase.ro.

Assignment 01.

Deadline - YYYY/MM/dd HH:mm: Current Year/10/27 23:50 GMT+2
Problem description: "Create multiple processes for communicating one each other, through UPD multicast/broadcast packages, in order to find out the best mark in the room. Each process has its own mark from 1 to 10 stored in command line for launching or it is stored in a file on the disk. The problem and the solutions have been explained in the lecture 01. You may develop the solution for this problem in Java/Kotlin or C++ / C# / node.js / Swift / Python." 
Upload and packiging: "The ZIP file has the source code files + *.sh/*.bat file for compiling and running the sample. The ZIP filename rule will have DAD2022_A01_SURNAME_FirstName1_FirstName2.zip". The upload is available through online.ase.ro..

Assignment 02.

Deadline - YYYY/MM/dd HH:mm: Current Year/10/27 23:50 GMT+2 
Problem description: "Build SNMP client program which is able to send SNMP traps in Java over HTTP or Websocket."

Upload and packiging: "ZIP file uploaded in SAKAI. The ZIP file has the PDF file that contains the steps + *.sh file for compiling and running the sample + *.java files with the source code + readme.txt which contains the configuration steps. The ZIP filename rule will have DAD2022_A02_SURNAME_FirstName1_FirstName2.zip". The upload is available through online.ase.ro

Assignment 03.

Deadline - YYYY/MM/dd HH:mm: Current Year/11/03 23:50 GMT+2
Problem description: "

3.1 Create a C program under Linux for adding to two arrays of float using IPC multi-process techniques - e.g. fork - https://www.geeksforgeeks.org/fork-system-call/ | https://www.includehelp.com/c-programs/c-fork-function-linux-example.aspx | https://stackoverflow.com/questions/15102328/how-does-fork-work

3.2 Create a C program under Linux for adding to two arrays of float using POSIX threads - https://www.cs.cmu.edu/afs/cs/academic/class/15492-f07/www/pthreads.html

3.3 Create a C++ '11/'17 program under Linux for adding to two arrays of float using objects from class thread - https://stackoverflow.com/questions/266168/simple-example-of-threading-in-c | https://www.geeksforgeeks.org/multithreading-in-cpp/ 

3.4 Create a C/C++ program under Linux for adding to two arrays of float using socket programming - https://gist.github.com/saxbophone/f770e86ceff9d488396c0c32d47b757e | https://www.geeksforgeeks.org/udp-server-client-implementation-c/  

3.5 Build client-server programs in Java. The client read the arrays from files and send them using Sockets and it transmits TCP packets. The servers receives the array and process them using multi-threading. The results are sent back asynchronous by the server to the clients by using Java NIO API. - Please see the discussion thread @ https://stackoverflow.com/questions/56747881/how-to-use-multiple-cores-in-the-java-non-blocking-i-o-nio-api"

Upload and packiging: "ZIP file uploaded in SAKAI. The ZIP file has the PDF file that contains the steps + *.sh file for compiling and running the sample + *.java files with the source code. The ZIP filename rule will have DAD_A03_SURNAME_FirstName1_FirstName2.zip". The upload is available through Moodle or SAKAI, please see E-Learning Solution section from this page.

Assignment 04.

Deadline - YYYY/MM/dd HH:mm: Current Year/11/10 23:50 GMT+2
Problem description: "Download laboratory/seminary with TCP HTTP Server server - proof of concept, in Linux Ubuntu (download the virtual machine from the current page - http://acs.ase.ro/dad), in order to solve HTTP requests for resources of type *.class. The URL in Internet browser - Mozilla Firefox/Google Chrome for http://ip_server:port_server/myfile.html will display the HTML file if exists, BUT when the browser asks http://ip_server:port_server/MyClassFile.class will display the string chars produced by method 'public String sampleDoGet();'. The one should modify method 'public String processInput(String theInput);' from class 'eu.ase.httpserver.HTTPSeminarProtocol' in order to handle through Java reflection the requiered task."
Upload and packiging: "ZIP file uploaded in SAKAI. The ZIP file has the Java source code *.java files + *.sh file for compiling and running the sample. The ZIP filename rule will have DAD_A04_SURNAME_FirstName1_FirstName2.zip". The upload is available through SAKAI, please see E-Learning Solution section from this page.

Assignment 05.

Deadline - YYYY/MM/dd HH:mm: Current Year/11/30 23:50 GMT+2
Problem description: "Download lecture and laboratory/seminary in Linux Ubuntu (download the virtual machine from this page - http://acs.ase.ro), in order to solve the following problem:
Develop the JSP and Java bean necessary for Apache Tomcat deploy that will receive via HTTP to vectors and will responde with the sum of the vectors. The Java bean from web will be the RMI client for 3 RMI servers objects, each RMI server object is registred in the same rmiregistry or in different rmiregistry program. Both versions, a) and b) are required.
Version a) - The RMI Server is developed in standard mode as in lecture sample (RMI01 and RMI03 directories).
Version b) - The RMI Server is developed in order to receive the bytecode of business logic (adding two vectors) in the same way as in ComputePi solution - ./downloads_readme/rmidocs/rmiDocsExec"
Upload and packiging: "ZIP file uploaded in SAKAI. The ZIP file has the Java source code *.java files + *.sh file for compiling and running the sample. The ZIP filename rule will have DAD_A05_SURNAME_FirstName1_FirstName2.zip". The upload is available through SAKAI, please see E-Learning Solution section from this page.

Assignment 06.

Deadline - YYYY/MM/dd HH:mm: Current Year/12/15 23:50 GMT+2 
Problem description: "Download activity with OpenMPI examples - proof of concept, in Linux Ubuntu (download the Docker container / virtual machine from the current page - http://acs.ase.ro/dad), in order to highlight 'broadcast', 'gather', 'scatter' on addition operation of two large arrays of integers - e.g. each having 100 000 000 elements. => each array elements occuping in RAM aprox. 300 - 400 MB" 
Upload and packiging: "ZIP file uploaded in SAKAI. The ZIP file has the C source code *.c files + *.sh file for compiling and running the sample. The ZIP filename rule will have DAD_A06_SURNAME_FirstName1_FirstName2.zip". The upload is available through SAKAI, please see E-Learning Solution section from this page.

Prerequisites

Fundamentals of Java SE plus C/C++  with JavaScript/ECMAScript/node.js + Networking + Linux/Windows OS | Optional – Python.

Target Group Profile

References

 

1.  Thomas Erl, Eric Barcelo Monroy, Cloud Computing: Concepts, Technology, and Architecture, Second Edition, Pearson, 2023

2. Justin Domingus, John Arundel, Cloud Native DevOps with Kubernetes, 2nd Edition, O'Reilly Media, Inc., 2022

3. Brendan Burns, "Designing Distributed Systems", Publisher: O'Reilly Media, Inc., Release Date: February 2018, ISBN: 9781491983645; with references to: Kelsey Hightower, Joe Beda, Brendan Burns, "Kubernetes: Up and Running, 2nd Edition", Publisher: O'Reilly Media, Inc., Release Date: October 2019, ISBN: 9781492046530

4. Kalpit Patel, John Carnell, "Spring Microservices in Action", Publisher: Manning Publications, Release Date: July 2017, ISBN: 9781617293986

5. Christina J. Hogan, Strata R. Chalup, Thomas A. Limoncelli, "Practice of Cloud System Administration, The: DevOps and SRE Practices for Web Services, Volume 2", Publisher: Addison-Wesley Professional, Release Date: September 2014, ISBN: 9780133478549 

6. George Coulouris, Jean Dollimore, Tim Kindberg, Gordon Blair, "Distributed Systems: Concepts and Design", Publisher Addison-Wesley; 5 edition (April 27, 2011), ISBN-10: 0132143011, ISBN-13: 978-0132143011 

7. Andrew S. Tanenbaum, Maarten Van Steen, "Distributed Systems: Principles and Paradigms (2nd Edition)", Publisher Prentice Hall; 2 edition (October 12, 2006), ISBN-10: 0132392275, ISBN-13: 978-0132392273

* My Experience and lectures/labs slides presentations, examples, virtual machines + your visual, kinetic and auditory memory + SAKAI e-Learning System PROVIDED by ISM – IT&C Security Master Program - http://ism.ase.ro | http://acs.ase.ro

*** DEPRECATED / OBSOLETE - IGNORE:

* The Linux Ubuntu 16.04 LTS OS VM as Oracle VirtualBox OVA Appliance for download - with GUI and Eclipse 2018-09 & IntelliJ IDEA 2018 as Java-IDE - 14GB @: 

https://drive.google.com/open?id=1kKZChf2qbYME9PtO3Gu9ZkecRoNykWsD

* Partial alternative to Oracle VirtualBox OVA, Docker Container Image with Linux Ubuntu 16.04 LTS for download - without GUI - 4GB @: 

https://hub.docker.com/r/critoma/ubuntu-java-node-py-dev/ 

Beyond the GUI and Eclipse & IntelliJ Java-IDE differences, both, the Oracle VirtualBox OVA appliance and the Docker Container Image contain:

Linux OS Ubuntu 16.04 x64 Xenial with: Java - Oracle JDK 8 and 9 and 11, Node.js 8.9 - JavaScript / ECMAScript, Python 2.7 and 3, Git, Apache ANT 1.10.1, Apache Maven 3.5.2, Gradle 4.5, Apache Tomcat 9.0.4, Jenkins Generic WAR 2.89.3 LTS.

E-Framework solutions for DAD lectures and labs were based till year 2017 on VMWare client virtual machines, and they have been provided by IT&C Security Master program - http://ism.ase.ro 

The OLD web address for download will be available for: 

- Linux Ubuntu 12 LTS (Can be updated to Ubuntu 14 with Java 8): with JDK 6 + Eclipse Indigo JEE + Netbeans + Apache Tomcat + JBOSS + GCC - Download 
- Alternatively, the latest versions of software may be installed by students in Ubuntu 14 VM (Oracle VM Box): JDK 8 + Eclipse LUNA + GCC - Download U14

Old Sakai instances: http://37.120.250.22:8088/portal / https://86.55.177.110:7443