What is DevOps?

devops-world

1. Definition from Wikipedia:

DevOps (a clipped compound of “development” and “operations”) is a set of software development practices[failed verification] that combines software development (Dev) with information technology operations (Ops) to shorten the systems development life cycle while delivering features, fixes, and updates frequently in close alignment with business objectives.

2. Definition from Webopedia

DevOps (development and operations) is an enterprise software development phrase used to mean a type of agile relationship between development and IT operations. The goal of DevOps is to change and improve the relationship by advocating better communication and collaboration between these two business units.

What is DevOps?

  1. DevOps is a culture that is followed by many big organizations. It is a combination of Development (software development) and Operations (software productions/IT operations).
  2. DevOps is an extended version of the Agile methodology. It is neither a tool or any technology, it is a culture or concept of behavior.
  3. DevOps defines a set of practices that are designed to overcome the communication and collaboration gap between the development, QA, and Operations team. It includes a Continuous Integration process with automated deployment.
  4. DevOps practices encourage the speed of delivering applications and services in the market.
  5. DevOps consists of 4 basic continuous processes in DevOps as follows:
  • Continuous Integration
  • Continuous Delivery
  • Continuous Testing
  • Continuous Monitoring

DevOps Life-cycle


i. Development
> In this DevOps stage the development of software takes place constantly. In this phase, the entire development process is separated into small development cycles. This benefits the DevOps team to speed up software development and delivery process.
ii. Testing
> QA team use tools like Selenium to identify and fix bugs in the new piece of code.
iii. Integration
> In this stage, new functionality is integrated with the prevailing code, and testing takes place. Continuous development is only possible due to continuous integration and testing.
iv. Deployment
> In this phase, the deployment process takes place continuously. It is performed in such a manner that any changes made any time in the code, should not affect the functioning of high traffic applications.
v. Monitoring
> In this phase, the operation team will take care of the inappropriate system behavior or bugs that are found in production.

Few of the benefits of DevOps are as follow

  • Speed
  • Rapid Delivery
  • Reliability
  • Scale
  • Improved collaboration
  • Security

The following are DevOps best practices

  • Continuous Integration
  • Continuous Delivery
  • Micro-services
  • Infrastructure as Code
  • Monitoring and Logging
  • Communication and Collaboration

storing multiple users and hosts in .my.cnf(MySQL)

When you run MySQL commands MySQL, mysqlcheck, mysqdump and psql, psqldump, etc; they will pick username & password from this file if you do not provide them as argument (-u and -p). It can save you time.

Of course, if you specify username and password explicitly as part of the command’s arguments, they will be used.

.my.cnf ( for MySQL client )

[clienthost1]   # Note: client + host1
user=myuser
password=mypass
database=dbname
host=server.location.com

Quick Install latest git from source

git-install-latest-from-src.sh

wget https://github.com/git/git/archive/master.zip
unzip master.zip && rm master.zip
cd master
make configure
./configure --prefix=/usr
make install

PHP 7 Installation in CentOS 7

PHP installation script

yum install -y http://rpms.remirepo.net/enterprise/remi-release-7.rpm
yum install -y yum-utils
# yum-config-manager --enable remi-php70
# yum-config-manager --enable remi-php71
# yum-config-manager --enable remi-php72
yum-config-manager --enable remi-php73
yum -y install php # php-mcrypt php-cli php-gd php-curl php-mysql php-ldap php-zip php-fileinfo 
php -v

storing multiple users and hosts in .pgpass(psql)

When you run MySQL commands MySQL, mysqlcheck, mysqdump and psql, psqldump etc; they will pick username & password from this file if you do not provide them as argument (-u and -p). It can save you time.

Of course, if you specify username and password explicitly as part of the command’s arguments, they will be used.

.pgpass ( for psql client )

test.net:5432:testdb:testuser:testpass
test1.net:5432:testdb1:testuser1:testpass1
test2.net:5432:testdb2:testuser2:testpass2

custom fail2ban filters

fail2Ban is a very handy tool to prevent a lot of unwanted traffic from consuming bandwidth on your servers. It’s a very small and relatively simple IDS Type Tool that comes with some predefined Filters to automatically lockout potentially dangerous or bandwidth-consuming type attacks.

Creating a Custom Filter

/etc/fail2ban/filter.d/custom.conf
[Definition]
 
badagents = 360Spider|ZmEu|Auto Spider 1.0|zgrab/[0-9]*\.[0-9a-zA-Z]*|Wget\(.*\)|MauiBot.*
 
failregex = ^.+?:\d+ <HOST> -.*"(GET|POST|HEAD).*HTTP.*(?:%(badagents)s)"$
 
ignoreregex =

Testing

fail2ban-regex /path-to-samples/sample.log /etc/fail2ban/filter.d/custom.conf

Jail example

[apache-custom]
enabled  = true
logpath  = /var/log/apache*/access.log
		   /var/log/apache*/ssl_access.log
action   = iptables-ipset-proto4[name=Custom, port=1010, protocol=tcp]
findtime = 86400
bantime  = -1
maxretry = 1

Exploring the Logrotate Configuration

Logrotate is a system utility that manages the automatic rotation and compression of log files. If log files were not rotated, compressed, and periodically pruned, they could eventually consume all available disk space on a system.

Logrotate’s configuration information can generally be found in two places:

/etc/logrotate.conf: this file contains some default settings and sets up the rotation for a few logs that are not owned by any system packages. It also uses an include statement to pull in configuration from any file in the /etc/logrotate.d directory.

/etc/logrotate.d/: this is where any packages you install that need help with log rotation will place their Logrotate configuration. On a standard install, you should already have files here for basic system tools like apt, dpkg, rsyslog and so on.

Example

/var/log/apt/history.log {
  rotate 12
  monthly
  compress
  missingok
  notifempty
  create 0640 www-data www-data
  sharedscripts
  postrotate
      systemctl reload example-app
  endscript
}

Explanation:

  • rotate 12: keep twelve old log files.
  • monthly: rotate once a month.
  • compress: compress the rotated files. this uses gzip by default and results in files ending in .gz. The compression command can be changed using the compresscmd option.
  • missingok: don’t write an error message if the log file is missing.
  • notifempty: don’t rotate the log file if it is empty.
  • create 0640 www-data www-data: this creates a new empty log file after rotation, with the specified permissions (0640), owner (www-data), and group (also www-data).
  • sharedscripts: this flag means that any scripts added to the configuration are run only once per run, instead of for each file rotated. Since this configuration would match two log files in the example-app directory, the script specified in postrotate would run twice without this option.
  • postrotate to endscript: this block contains a script to run after the log file is rotated. In this case we’re reloading our example app. This is sometimes necessary to get your application to switch over to the newly created log file. Note that postrotate runs before logs are compressed. Compression could take a long time, and your software should switch to the new logfile immediately. For tasks that need to run after logs are compressed, use the lastaction block instead.

MySQL – install, configure, create user and database with permissions

MySQL is an open-source relational database management system. Its name is a combination of “My”, the name of co-founder Michael Widenius’s daughter, and “SQL”, the abbreviation for Structured Query Language.

Download and Install MySQL from the following link

https://dev.mysql.com/downloads/

rpm -ih mysql80-community-release-el7-3.noarch.rpm
yum update -y
yum install -y mysql-server

Configure

systemctl enable mysqld
systemctl start mysqld
mysql_secure_installation
# default password
grep -oP 'temporary password(.*): \K(\S+)' /var/log/mysqld.log

Create user and database with permissions

mysql -u root -p
# mysql > 
CREATE DATABASE dbname;
CREATE USER 'myuser'@'localhost' IDENTIFIED BY 'mypass';
GRANT ALL PRIVILEGES ON dbname.* TO 'myuser'@'%';
FLUSH PRIVILEGES;

Starting with MySQL 8 you no longer can (implicitly) create a user using the GRANT command. Use CREATE USER instead, followed by the GRANT statement:

CREATE USER 'root'@'%' IDENTIFIED BY 'root';
GRANT ALL PRIVILEGES ON . TO 'root'@'%' WITH GRANT OPTION;