Linux Cloud Technologies 2013

  Build the cloud on Linux!  This year looks very promising for Linux when it comes to building your private cloud using open source technologies.  Finally, Linux-based software and application

Read More
Linux Cloud Technologies 2013

Modifying TP-Link TL-WN722N v2 driver

by on July 17, 2017 at 5:54 pm

I recently purchased a TP-Link wifi dongle for my desktop.  I have previously purchased TP-Link products and have enjoyed out of the box support.  Unfortunately for me, the latest version (v2.1 or v2.0) updated the chipset to Realtek 8188eus aka rtl8188eu.  Additionally, this product ships with the VendorID: 0×2357, ProductID: 0x010c.  These ID’s are what inform the kernel which driver to load to interface with the device.  Unfortunately, this Vendor and Product ID combination is not found in any of the existing kernel drivers.

UPDATE:  Are you just looking for a working driver?  Give https://github.com/lwfinger/rtl8188eu a try.  LW Finger is a kernel maintainer, the software should be much better than mine below.

My first step in the journey to getting this dongle to work was to download the driver’s from TP link’s site directly:  http://www.tp-link.com/us/download/TL-WN722N.html#Driver

The drivers are labeled as “Linux Kernel 2.6.18~3.19.3.” on that site.  Wow, that’s old.  I’m running Ubuntu 16.04 LTS with kernel 4.4.0.  This might not work out.  Well, as it happens, I download the tarball, which is actually just a tar of C source files.  Score.  On the plus side, the folder is labeled for kernel 4.3.  I make a few modifications and now it compiles and I can use my dongle: https://github.com/michaelgugino/rtl8188EUS

Now that I know that the device actually works on Linux, and the driver code seems to be a somewhat old, I start browsing the linux kernel source looking for any compatible drivers.  I first discovered drivers/net/wireless/realtek/rtl8xxxu.  This claims to have some support for 8188eu devices, but I don’t see any devices with that chipset being used in that driver code, but the preliminary support is there for adding devices in the future (which is outside of my skill level and time commitment for this project).  After trying to force the Vendor and Product ID into that driver, I get an error in dmesg stating the eFUSE offset is wrong.  So, the existing devices that I tried in that driver weren’t 100% compatible.

After a little more grep’ing, I stumbled along drivers/staging/rtl8188eu.  Bingo.  This supports USB devices and looks like exactly what I need.  I chop a quick patch and recompile that driver module against my kernel, load it, and it’s working.  Perfect.  Only problem is performance doesn’t seem to be on par with the vendor supplied driver.  This might be due to the fact I’m using a much older kernel, 4.4.0, and performance may be fixed upstream.  In any case, modifying the source for either 4.4.0 or a newer kernel should be the same one-line change.  Patch here: http://driverdev.linuxdriverproject.org/pipermail/driverdev-devel/2017-July/108601.html

in Uncategorized

Deploying Apache Virtual Hosts using Puppet on CentOS 6

by on October 13, 2014 at 7:03 pm

Scaling a website to serve thousands or even tens of thousands of users simultaneously is a challenge often best tackled by horizontal scaling – distributing workloads across dozens or even hundreds of servers. As a tool for preparing servers for that task, Puppet offers low deployment costs, ease of use, and automated configuration management.

After a successful deployment of a new hardware farm, how can you assure a static configuration across your entire environment? Puppet addresses that problem. Let’s see how to install Puppet and use it to deploy, as an example, an Apache web server virtual host on CentOS 6. This tutorial shows how to deploy virtual hosts on only one server, but the same steps can be replicated to manage many servers. I’ll assume you’re familiar with the Linux command line, basic networking concepts, and using Apache.


in Uncategorized
Ad goes here

AWS: Use instance role credentials to query ec2 API

by on September 16, 2014 at 3:58 pm

I was having some issues including a token in v4 signing requests using the ec2 query API.  With the help of the excellent AWS support, I know have a working example based on the documentation provided by Amazon.

# AWS Version 4 signing example

# EC2 API (DescribeRegions)

# See: http://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html
# This version makes a GET request and passes the signature
# in the Authorization header.
import sys, os, base64, datetime, hashlib, hmac, json
import requests # pip install requests

# ************* REQUEST VALUES *************
method = 'GET'
service = 'ec2'
host = 'ec2.amazonaws.com'
region = 'us-east-1'
endpoint = 'https://ec2.amazonaws.com'
request_parameters = 'Action=DescribeRegions&Version=2013-10-15'

# Get the Role information and credentials
r = requests.get('');
role = r.text
r = requests.get('' + role);
decoded_data = json.loads(r.text)
access_key = decoded_data['AccessKeyId']
secret_key = decoded_data['SecretAccessKey']
token = decoded_data['Token']

# Key derivation functions. See:
# http://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-python
def sign(key, msg):
 return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()

def getSignatureKey(key, dateStamp, regionName, serviceName):
 kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
 kRegion = sign(kDate, regionName)
 kService = sign(kRegion, serviceName)
 kSigning = sign(kService, 'aws4_request')
 return kSigning

# Create a date for headers and the credential string
t = datetime.datetime.utcnow()
amzdate = t.strftime('%Y%m%dT%H%M%SZ')
datestamp = t.strftime('%Y%m%d') # Date w/o time, used in credential scope
# ************* TASK 1: CREATE A CANONICAL REQUEST *************
# http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html

# Step 1 is to define the verb (GET, POST, etc.)--already done.

# Step 2: Create canonical URI--the part of the URI from domain to query
# string (use '/' if no path)
canonical_uri = '/'

# Step 3: Create the canonical query string. In this example (a GET request),
# request parameters are in the query string. Query string values must
# be URL-encoded (space=%20). The parameters must be sorted by name.
# For this example, the query string is pre-formatted in the request_parameters variable.
canonical_querystring = request_parameters

# Step 4: Create the canonical headers and signed headers. Header names
# and value must be trimmed and lowercase, and sorted in ASCII order.
# Note that there is a trailing \n.
canonical_headers = 'host:' + host + '\n' + 'x-amz-date:' + amzdate + '\n'

# Step 5: Create the list of signed headers. This lists the headers
# in the canonical_headers list, delimited with ";" and in alpha order.
# Note: The request can include any headers; canonical_headers and
# signed_headers lists those that you want to be included in the
# hash of the request. "Host" and "x-amz-date" are always required.
signed_headers = 'host;x-amz-date'

# Step 6: Create payload hash (hash of the request body content). For GET
# requests, the payload is an empty string ("").
payload_hash = hashlib.sha256('').hexdigest()

# Step 7: Combine elements to create create canonical request
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash
# ************* TASK 2: CREATE THE STRING TO SIGN*************
# Match the algorithm to the hashing algorithm you use, either SHA-1 or
# SHA-256 (recommended)
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'
string_to_sign = algorithm + '\n' + amzdate + '\n' + credential_scope + '\n' + hashlib.sha256(canonical_request).hexdigest()
# ************* TASK 3: CALCULATE THE SIGNATURE *************
# Create the signing key using the function defined above.
signing_key = getSignatureKey(secret_key, datestamp, region, service)

# Sign the string_to_sign using the signing_key
signature = hmac.new(signing_key, (string_to_sign).encode('utf-8'), hashlib.sha256).hexdigest()
# ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************
# The signing information can be either in a query string value or in
# a header named Authorization. This code shows how to use a header.
# Create authorization header and add to request headers
authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' + 'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature

# The request can include any headers, but MUST include "host", "x-amz-date",
# and (for this scenario) "Authorization". "host" and "x-amz-date" must
# be included in the canonical_headers and signed_headers, as noted
# earlier. Order here is not significant.
# Python note: The 'host' header is added automatically by the Python 'requests' library.
headers = {'x-amz-date':amzdate , 'Authorization':authorization_header, 'X-Amz-Security-Token':token}
# ************* SEND THE REQUEST *************
request_url = endpoint + '?' + canonical_querystring

print '\nBEGIN REQUEST++++++++++++++++++++++++++++++++++++'
print 'Request URL = ' + request_url
r = requests.get(request_url, headers=headers)

print '\nRESPONSE++++++++++++++++++++++++++++++++++++'
print 'Response code: %d\n' % r.status_code
print r.text

Hopefully you find this useful.

in Uncategorized

, ,

Join Ubuntu 14.04 to Active Directory Domain using realmd

by on April 29, 2014 at 9:15 pm

This proved to be a difficult task.  I spent several hours scouring the internet for various bugs in this process to little avail.  I’m going to summarize what I did to actually get this puppy up and running.

Started with a clean install of Ubuntu 14.04 LTS Server Edition.  Pointed my DNS to my AD controller.


in Uncategorized

CentOS 6 Google App Engine Python Development with Eclipse

by on November 26, 2013 at 5:34 pm

With more and more companies moving applications to the cloud, Google App Engine makes a lot of sense.  GAE is a Platform as a Service (PaaS) product offered which runs on Google’s infrastructure.  Some of the touted capabilities are seamless, limitless, and completely automated application scaling.  In this article, you’ll learn how to setup a basic development environment for Google App Engine’s Python SDK on CentOS 6 using PyDev and Eclipse.


, , , ,

Join CentOS 6 to Active Directory Domain

by on October 1, 2013 at 4:03 pm

Joining CentOS 6 or Red Hat Enterprise Linux 6 to an Active Directory Domain is relatively simple.  While Active Directory is proprietary software developed by Microsoft, it’s fairly ubiquitous in medium and large environments, thus integrating Linux and Windows services is very common in this day and age.  DNS has to be working properly.  You should be able to resolve mydomain.com using DNS.

First, we need to install winbind.  This is the Samba service that integrates users, passwords, and other important functions with Active Directory.

yum install samba-winbind

That command should install any and all dependencies necessary.  Another step is to install software necessary for initializing Kerberos tickets.  While not strictly necessary to join the Domain initially (I believe), it makes troubleshooting a little easier.

yum install krb5-workstation

After those two packages are installed, you can run authconfig-tui to automatically setup pam and other important config files.  See the screen shots below for example settings.


The above selections are appropriate.  Use fingerprint reader is not needed unless your workstation has a fingerprint reader.


This stage is very important.  Security model should be set to ADS.  Domain should be the name of the domain without the top level domain.  If your domain looks like my.domain.com, then you should put “MY” in this field.  Domain controllers are the FQDN for each domain controller you wish your system to use.  Unlike Windows, these are not automatically discovered by CentOS or RHEL 6.  Separate each domain controller by a space.  ADS REALM should be the full name of your Domain in ALL CAPS.  Template shell can be whichever you choose.  If you want to enable domain users the ability to log in by default, select one of the shells.  If you want to disable ssh/local login by default, select /sbin/nologin.

Next, select Join Domain and enter your Domain Admin username and password in the boxes provided.  You should enter just the username, do not enter any domain information here.

, , , , ,

Join Fedora 19 to Active Directory Domain using realmd

by on September 23, 2013 at 4:24 pm

For years, Linux administrators have been successfully using Samba winbind to integrate Linux with Active directory.  While configuring a Linux host to join an Active Directory Domain is pretty simple, it still involves editing a few configuration files manually in most cases.  The new software, realmd, changes all of that, and makes joining a Linux host to an Active Directory Domain easier than ever before!


, ,

SQL Server insert with sub queries

by on July 9, 2013 at 1:51 pm

Extremely awesome SQL query I wrote.

use my_db


ltrim(rtrim(str(b.my_id)))+ltrim(rtrim(str(b.date))) pid_date,
p.column1+p.column2 ssdd,
COUNT (*) Amount,
select COUNT(*) from amounts b1
where b1.flags = '1'
and b1.my_id = b.my_id
and b1.date = b.date
) Deleted,
select SUM(m.count1)
from mail_counts m
where m.my_id = b.my_id
and m.date = b.date
) Counts

 from amounts b

inner join pusers p
on b.my_id = p.my_id

where (b.date < '20130701' and b.date >= '20120101')
and not exists (
select *
FROM dbo.my_archive mt
where mt.pid_date = ltrim(rtrim(str(b.my_id)))+ltrim(rtrim(str(b.date)))
and p.column1 != 'TS'

group by b.date, b.my_id, p.first_name, p.last_name, p.column1+p.column2

in Uncategorized


Top 10 useful linux cli tools

by on April 11, 2013 at 3:42 pm

Here are some tools I use on a fairly frequent basis, which may or may not be installed by default in your distribution.  I highly recommend installing these tools during the provisioning stage of your system/environment because repositories aren’t always reachable on production systems.  Furthermore, installing software on a running critical production system is often a tightly controlled process.


in Uncategorized

Python Backup Script

by on February 21, 2013 at 10:23 pm

Part one, take the first full backup. At the present time, this code is still under development and should not be used on a production machine. However, I am posting it here for reference.

Eventually, this code is going to be included in a backup client I am developing that will interface with glusterfs and Amazon S3 storage.

Currently, this code is tested to run on Python v2.7.4 on a Fedora 18 machine. With all three python files, and any number of properly defined job xml files in the jobs.d/ directory, these scripts are currently functional.


in Uncategorized

, , , ,