Featured

Linux Cloud Technologies 2013

  Build the cloud on Linux!  This year looks very promising for Linux when it comes to building your private cloud using open source technologies.  Finally, Linux-based software and application

Read More
Linux Cloud Technologies 2013

Deploying Apache Virtual Hosts using Puppet on CentOS 6

0
by on October 13, 2014 at 7:03 pm

Scaling a website to serve thousands or even tens of thousands of users simultaneously is a challenge often best tackled by horizontal scaling – distributing workloads across dozens or even hundreds of servers. As a tool for preparing servers for that task, Puppet offers low deployment costs, ease of use, and automated configuration management.

After a successful deployment of a new hardware farm, how can you assure a static configuration across your entire environment? Puppet addresses that problem. Let’s see how to install Puppet and use it to deploy, as an example, an Apache web server virtual host on CentOS 6. This tutorial shows how to deploy virtual hosts on only one server, but the same steps can be replicated to manage many servers. I’ll assume you’re familiar with the Linux command line, basic networking concepts, and using Apache.

(more…)

in Uncategorized

AWS: Use instance role credentials to query ec2 API

0
by on September 16, 2014 at 3:58 pm

I was having some issues including a token in v4 signing requests using the ec2 query API.  With the help of the excellent AWS support, I know have a working example based on the documentation provided by Amazon.


# AWS Version 4 signing example

# EC2 API (DescribeRegions)

# See: http://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html
# This version makes a GET request and passes the signature
# in the Authorization header.
import sys, os, base64, datetime, hashlib, hmac, json
import requests # pip install requests

# ************* REQUEST VALUES *************
method = 'GET'
service = 'ec2'
host = 'ec2.amazonaws.com'
region = 'us-east-1'
endpoint = 'https://ec2.amazonaws.com'
request_parameters = 'Action=DescribeRegions&Version=2013-10-15'

# Get the Role information and credentials
r = requests.get('http://169.254.169.254/latest/meta-data/iam/security-credentials');
role = r.text
r = requests.get('http://169.254.169.254/latest/meta-data/iam/security-credentials/' + role);
decoded_data = json.loads(r.text)
access_key = decoded_data['AccessKeyId']
secret_key = decoded_data['SecretAccessKey']
token = decoded_data['Token']

# Key derivation functions. See:
# http://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-python
def sign(key, msg):
 return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()

def getSignatureKey(key, dateStamp, regionName, serviceName):
 kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
 kRegion = sign(kDate, regionName)
 kService = sign(kRegion, serviceName)
 kSigning = sign(kService, 'aws4_request')
 return kSigning

# Create a date for headers and the credential string
t = datetime.datetime.utcnow()
amzdate = t.strftime('%Y%m%dT%H%M%SZ')
datestamp = t.strftime('%Y%m%d') # Date w/o time, used in credential scope
# ************* TASK 1: CREATE A CANONICAL REQUEST *************
# http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html

# Step 1 is to define the verb (GET, POST, etc.)--already done.

# Step 2: Create canonical URI--the part of the URI from domain to query
# string (use '/' if no path)
canonical_uri = '/'

# Step 3: Create the canonical query string. In this example (a GET request),
# request parameters are in the query string. Query string values must
# be URL-encoded (space=%20). The parameters must be sorted by name.
# For this example, the query string is pre-formatted in the request_parameters variable.
canonical_querystring = request_parameters

# Step 4: Create the canonical headers and signed headers. Header names
# and value must be trimmed and lowercase, and sorted in ASCII order.
# Note that there is a trailing \n.
canonical_headers = 'host:' + host + '\n' + 'x-amz-date:' + amzdate + '\n'

# Step 5: Create the list of signed headers. This lists the headers
# in the canonical_headers list, delimited with ";" and in alpha order.
# Note: The request can include any headers; canonical_headers and
# signed_headers lists those that you want to be included in the
# hash of the request. "Host" and "x-amz-date" are always required.
signed_headers = 'host;x-amz-date'

# Step 6: Create payload hash (hash of the request body content). For GET
# requests, the payload is an empty string ("").
payload_hash = hashlib.sha256('').hexdigest()

# Step 7: Combine elements to create create canonical request
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash
# ************* TASK 2: CREATE THE STRING TO SIGN*************
# Match the algorithm to the hashing algorithm you use, either SHA-1 or
# SHA-256 (recommended)
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'
string_to_sign = algorithm + '\n' + amzdate + '\n' + credential_scope + '\n' + hashlib.sha256(canonical_request).hexdigest()
# ************* TASK 3: CALCULATE THE SIGNATURE *************
# Create the signing key using the function defined above.
signing_key = getSignatureKey(secret_key, datestamp, region, service)

# Sign the string_to_sign using the signing_key
signature = hmac.new(signing_key, (string_to_sign).encode('utf-8'), hashlib.sha256).hexdigest()
# ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************
# The signing information can be either in a query string value or in
# a header named Authorization. This code shows how to use a header.
# Create authorization header and add to request headers
authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' + 'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature

# The request can include any headers, but MUST include "host", "x-amz-date",
# and (for this scenario) "Authorization". "host" and "x-amz-date" must
# be included in the canonical_headers and signed_headers, as noted
# earlier. Order here is not significant.
# Python note: The 'host' header is added automatically by the Python 'requests' library.
headers = {'x-amz-date':amzdate , 'Authorization':authorization_header, 'X-Amz-Security-Token':token}
# ************* SEND THE REQUEST *************
request_url = endpoint + '?' + canonical_querystring

print '\nBEGIN REQUEST++++++++++++++++++++++++++++++++++++'
print 'Request URL = ' + request_url
r = requests.get(request_url, headers=headers)

print '\nRESPONSE++++++++++++++++++++++++++++++++++++'
print 'Response code: %d\n' % r.status_code
print r.text

Hopefully you find this useful.

in Uncategorized

, ,

Ad goes here

Join Ubuntu 14.04 to Active Directory Domain using realmd

19
by on April 29, 2014 at 9:15 pm

This proved to be a difficult task.  I spent several hours scouring the internet for various bugs in this process to little avail.  I’m going to summarize what I did to actually get this puppy up and running.

Started with a clean install of Ubuntu 14.04 LTS Server Edition.  Pointed my DNS to my AD controller.

(more…)

in Uncategorized

CentOS 6 Google App Engine Python Development with Eclipse

0
by on November 26, 2013 at 5:34 pm

With more and more companies moving applications to the cloud, Google App Engine makes a lot of sense.  GAE is a Platform as a Service (PaaS) product offered which runs on Google’s infrastructure.  Some of the touted capabilities are seamless, limitless, and completely automated application scaling.  In this article, you’ll learn how to setup a basic development environment for Google App Engine’s Python SDK on CentOS 6 using PyDev and Eclipse.

(more…)

, , , ,

Join CentOS 6 to Active Directory Domain

9
by on October 1, 2013 at 4:03 pm

Joining CentOS 6 or Red Hat Enterprise Linux 6 to an Active Directory Domain is relatively simple.  While Active Directory is proprietary software developed by Microsoft, it’s fairly ubiquitous in medium and large environments, thus integrating Linux and Windows services is very common in this day and age.  DNS has to be working properly.  You should be able to resolve mydomain.com using DNS.

First, we need to install winbind.  This is the Samba service that integrates users, passwords, and other important functions with Active Directory.

yum install samba-winbind

That command should install any and all dependencies necessary.  Another step is to install software necessary for initializing Kerberos tickets.  While not strictly necessary to join the Domain initially (I believe), it makes troubleshooting a little easier.

yum install krb5-workstation

After those two packages are installed, you can run authconfig-tui to automatically setup pam and other important config files.  See the screen shots below for example settings.

authconfig-tui

The above selections are appropriate.  Use fingerprint reader is not needed unless your workstation has a fingerprint reader.

authconfig-tui

This stage is very important.  Security model should be set to ADS.  Domain should be the name of the domain without the top level domain.  If your domain looks like my.domain.com, then you should put “MY” in this field.  Domain controllers are the FQDN for each domain controller you wish your system to use.  Unlike Windows, these are not automatically discovered by CentOS or RHEL 6.  Separate each domain controller by a space.  ADS REALM should be the full name of your Domain in ALL CAPS.  Template shell can be whichever you choose.  If you want to enable domain users the ability to log in by default, select one of the shells.  If you want to disable ssh/local login by default, select /sbin/nologin.

Next, select Join Domain and enter your Domain Admin username and password in the boxes provided.  You should enter just the username, do not enter any domain information here.

, , , , ,

Join Fedora 19 to Active Directory Domain using realmd

18
by on September 23, 2013 at 4:24 pm

For years, Linux administrators have been successfully using Samba winbind to integrate Linux with Active directory.  While configuring a Linux host to join an Active Directory Domain is pretty simple, it still involves editing a few configuration files manually in most cases.  The new software, realmd, changes all of that, and makes joining a Linux host to an Active Directory Domain easier than ever before!

(more…)

, ,

SQL Server insert with sub queries

0
by on July 9, 2013 at 1:51 pm

Extremely awesome SQL query I wrote.


use my_db

INSERT INTO
 dbo.my_archive

select
ltrim(rtrim(str(b.my_id)))+ltrim(rtrim(str(b.date))) pid_date,
b.my_id,
b.date,
p.column1+p.column2 ssdd,
p.last_name,
p.first_name,
COUNT (*) Amount,
(
select COUNT(*) from amounts b1
where b1.flags = '1'
and b1.my_id = b.my_id
and b1.date = b.date
) Deleted,
(
select SUM(m.count1)
from mail_counts m
where m.my_id = b.my_id
and m.date = b.date
) Counts

 from amounts b

inner join pusers p
on b.my_id = p.my_id

where (b.date < '20130701' and b.date >= '20120101')
and not exists (
select *
FROM dbo.my_archive mt
where mt.pid_date = ltrim(rtrim(str(b.my_id)))+ltrim(rtrim(str(b.date)))
)
and p.column1 != 'TS'

group by b.date, b.my_id, p.first_name, p.last_name, p.column1+p.column2

in Uncategorized

,

Top 10 useful linux cli tools

3
by on April 11, 2013 at 3:42 pm

Here are some tools I use on a fairly frequent basis, which may or may not be installed by default in your distribution.  I highly recommend installing these tools during the provisioning stage of your system/environment because repositories aren’t always reachable on production systems.  Furthermore, installing software on a running critical production system is often a tightly controlled process.

(more…)

in Uncategorized

Python Backup Script

0
by on February 21, 2013 at 10:23 pm

Part one, take the first full backup. At the present time, this code is still under development and should not be used on a production machine. However, I am posting it here for reference.

Eventually, this code is going to be included in a backup client I am developing that will interface with glusterfs and Amazon S3 storage.

Currently, this code is tested to run on Python v2.7.4 on a Fedora 18 machine. With all three python files, and any number of properly defined job xml files in the jobs.d/ directory, these scripts are currently functional.

(more…)

in Uncategorized

, , , ,

Backup MySQL database

0
by on January 16, 2013 at 6:44 pm

Today, we’re going to be covering the 3 most important rules for any Linux System Administrator:  Backups, Backups, Backups.

That’s right, if you don’t have backups, you have failed in your duties.  Every single thing else you may have done to secure your system cannot replace the need for backups.  Systems get cracked, hard drives fail, CPUs fail, RAM fails, password are forgotten, files get rm’d by mistake, patches break systems.

This article is about creating a simple backup script to backup your MySQL databases to an offsite location, complete with cron jobs, and encryption.

(more…)

, , , ,