Wednesday, December 15, 2010

Python Basic HTTPS webservice

The basic webservice module in my previous posting does not support https or ssl encryption.

Here is how I finally managed to work it out:


# Basic https web server

import socket, ssl

host = ''
port = 8080

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind((host, port))
sock.listen(1)

while 1:
osock, caddr = sock.accept()
csock = ssl.wrap_socket(osock,
server_side=True,
certfile="servercert.cer",
keyfile="serverkey.key",
ssl_version=ssl.PROTOCOL_SSLv23)

cfile = csock.makefile('rw', 0)

# Protocol exchange - read request
msg = "GO AWAY!"
while 1:
line = cfile.readline().strip()
if line == "GET /echotest HTTP/1.1":
msg = "ECHOTEST OK"
if line == '':
cfile.write("HTTP/1.0 200 OK\n\n")
cfile.write("<head><title>Eh?</title></head>")
cfile.write("<h1>%s</h1>" % msg)
cfile.close()
csock.close()
osock.close()
break

Monday, November 8, 2010

Compile Pana (Amarok 1.4 clone) on Fedora 14 x86_64

After meeting all the various dependencies, I had to make one small actual code change to the source code. I would have liked to submit it on the Pana website but could not find a way to do that.

Anyway here is the patch.

[dave@fedora junk]$ diff [source tree]/pana/src/osd.h [source tree]/pana/src/osd.h.original
40c40
< void show( const QString &text, QImage newImage = QImage() );
---
> void show( const QString &text, QImage newImage = QImage::QImage() );

Thursday, October 14, 2010

Python webservice that executes local commands

There are a few different options when it comes to managing server-side scripting on a web site. Usually folks use php or perl and even python in many occasions.

This blog post is about using python to execute code locally on the server in response to http GET requests.

So far you are thinking so what? You are already crafting your comment and it is saying something like, "Google mod_python" or "Google mod_perl". You are right, the best way to do CGI is via mod_perl, mod_php or mod_perl. The problem is user access and chroot.

Apache will execute server side scripts as the user / group defined in the main httpd.conf. In my case: apache / apache.
Apache will also assume a document root of /var/www/ for scripts (on a Centos 5.5 box) even if the userdir module is in use.

My problem was: How to get apache to execute scripts as dave:dave on doc root = /home/dave/. It was critical to get this working because the scripts in question interact with the .gnupg/pubkeyring and .gnupg/seckeyring files under /home/dave/.gnupg/.

Basically, I was making some kind of web based PGP key server. A web based gui for remote users to manage keys.

In the end I settled for python and the BaseHTTPServer.

First of all a simple class that will accept a shell command, execute it and return the stdout.


import popen2

class MyShellCommand:

"""execute a command, capture stdout and return it."""
def __callShellCmd(self, cmd):

stdout, stdin = popen2.popen2(command)
data = ""
while True:
c = stdout.read(1)
if( c ):
data += c
else:
break
return data

"""concrete example"""
def getPublicGPGKey(self, keyid):

"""TODO: Add logic to validate key id..."""
command = "gpg -a --export '%s'" % keyid
return self.__callShellCmd(command)


Now that we have a utility to retrieve public keys from the gpg keyring, lets call it from a webservice that is owned and operated by user:group, dave:dave.


import MyShellCommand
import time
import BaseHTTPServer, cgi

"""Configure host ip and port to listen on. Use high port for non root users."""
HOST_NAME = '127.0.0.1'
PORT_NUMBER = 8080

class MyHandler(BaseHTTPServer.BaseHTTPRequestHandler):

def do_HEAD(s):
s.send_response(200)
s.send_header("Content-type", "text/html")
s.end_headers()

def do_GET(s):
s.send_response(200)
s.send_header("Content-type", "text/html")
s.end_headers()

"""Get the path and find the parameters"""
path, query_string = s.path.split('?', 1)
params = dict(cgi.parse_qsl(query_string))

"""create a shell object"""
shell = MyShellCommand()

"""Validate the call being made"""
if path == '/publickey':
"""Validate the parameters"""
if params.has_key('id'):
s.wfile.write('%s' % shell.getPublicGPGKey(params['id']))

"""I dont like descriptive errors."""
else: s.wfile.write('An error occurred.')
else: s.wfile.write('An error occurred.')



if __name__ == '__main__':
server_class = BaseHTTPServer.HTTPServer
httpd = server_class((HOST_NAME, PORT_NUMBER), MyHandler)
print time.asctime(), "Server Starts - %s:%s" % (HOST_NAME, PORT_NUMBER)
try:
httpd.serve_forever()
except KeyboardInterrupt:
pass
httpd.server_close()
print time.asctime(), "Server Stops - %s:%s" % (HOST_NAME, PORT_NUMBER)


If you execute the above script, you will have a working webservice that responds nicely to only one specific set of GET data. Call it with a URL like this:


http://127.0.0.1:8080/publickey?id=mykeyname

Friday, October 1, 2010

Use PHP to perform an LDAP Bind to Windows Active Directory for User Authentication

This example shows some basic LDAP bind lookups on a Windows 2003 active directory server.

The sequence is:

  • Connect 
  • Bind using privileged user 
  • search for 'dn' of supplied credentials
  • re-bind using this dn and password of supplied credentials
  • unbind

The per-requisite is that php_ldap modules are loaded and compiled.  These are fairly standard now days.  In fedora it was just a matter of executing:



# yum install php-ldap

Anyway:  Here is the example.  It's farily well commented so should be a simple matter to make work in your own environment.



<?php
// example.php
//
// David Latham @ 2010
// david-latham.blogspot.com
//
// This code cannot be executed on the same server as AD is installed on!!!
//
// Active Directory has an Organizational Unit of "_Test_Users" with 3 users loaded
// into it.
// user1 | User One
// user2 | User Two
// ldapbind | ldap bind
//
// execute with php -q example.php

// credentials to test
$user='user2@example.local';
// user='user2'; //Use this one to test binding without an email address.
$pass='2wsx#EDC';

// AD server details
$ldap_server = "192.168.122.231";
$dn = "OU=_Test_Users,DC=example,DC=local";

// Connect
$ad = ldap_connect($ldap_server); //Must be a real active directory server
//(WIN2003 used in this example)

// Set some variables
ldap_set_option($ad, LDAP_OPT_PROTOCOL_VERSION, 3);
ldap_set_option($ad, LDAP_OPT_REFERRALS, 0);

// Bind to the ldap directory.
// Here we are binding using a fixed user who has privileges to bind to the
// active directory. This user also has a fixed password. It can not change
// otherwise this script will break.


$bd = @ldap_bind($ad,"ldap user","pa$$word")
or die("Couldn't bind to AD!");

// Search the directory for a distinguished name (dn) using the supplied
// credentials.
//
// If $user is an email address then use 'mail' otherwise use 'sAMAccountName'

if( strpos($user, '@') > 0 ) {
$field='mail';
} else {
$field='sAMAccountName';
}
echo "Searching AD using ...".$field."\n";

$result = @ldap_search($ad, $dn, "(".$field."=".$user.")", array('dn'));

// Create result set
$entries = @ldap_get_entries($ad, $result);

// Show an array of result
print_r($entries);

// If we have a result then we should 're-bind' using the destinguished name found
// in the search.
if ( $entries[0] ) {
if(@ldap_bind($ad, $entries[0]['dn'], $pass)) {
// If bind was successful then show some data. This is user
// authentication success.
echo "found: ".$entries[0]['dn'];
} else {
// Bind failed so this would equate to user authentication failure.
echo "Failed to bind.";
}
} else {
// Failure to search would indicate a problem with your $dn variable.
echo "failed to search";
}

//never forget to unbind!
ldap_unbind($ad);

?>

the result might look something like this:



[dave@fedora php-ldap]$ php -q example.php
Searching AD using ...mail
Array
(
[count] => 1
[0] => Array
(
[count] => 0
[dn] => CN=user2,OU=_Test_Users,DC=example,DC=local
)

)
found: CN=user2,OU=_Test_Users,DC=example,DC=local

Have fun authenticating your users against your work active directory. You will:

  • Please your security manager

  • Avoid the headache of managing passwords password compliance

  • Write safer applications in terms of locking them down

  • Feel good about yourself

Wednesday, June 16, 2010

Images from web in python gui

I have an application that I use to monitor data produced by a webservice. This webservice produces a new image ( a .png graph to be precise ) every 15 seconds. Basically I have four cron jobs that execute every minute. Three of these are configured to sleep for 15s, 30s and 45s respectivley. So that's one way to make cron do stuff more often than once a minute.

Anyway, I have been using a web browser and some fancy ajax type functions to continuously display this graph and some other JSON data from the webserver every 15 seconds forever. We have a group of people who have the job of watching my graphs and data among a whole bunch of other stuff in case they might provide some alert to a problem.

Anyway, I have been wondering, of late, about the reliablility of using a web browser for monitoring. The web browser is not refreshing itself and so I am a little worried that it just kind of get's stuck at times. I think it looses the AJAXIAN plot so to speak. This is why I started making a python gui using the following python modules:
  • httplib
  • json
  • urrlib
  • wx
  • logging
I will only show three classes here for reference purposes.

There is also a threading module required. This bit of handy code is at the bottom and I must apologies to the author of this code as I failed to take a note of his name / URL and so can not reference the source.

This is the Web client class that I use to retrieve the http / JSON content and the image content from my webserver.

#!/usr/bin/env python

import httplib
import urllib

class HTTP_Client(httplib.HTTPConnection):

def __init__(self, url):
httplib.HTTPConnection.__init__(self, url)


def getPage(self, page):
self.request("GET", "/%s" % page)
r = self.getresponse()
data = r.read()
if r.status == 200 and r.reason == "OK":
return urllib.base64(data)
else:
return "E Fetching %s: %s %s" % (page, r.status, r.reason)

if __name__ == "__main__":
hc = HttpClient("www.python.org")
print hc.getPage("parrot.spam")
print hc.getPage("index.html")


This is an example of how to display an image on a panel using the wx gui toolkit. I think this is a mostly coding by google but it's not a big leap to make it useful. I believe my real program uses the _Image.SetBitmap(_Bitmap) method to update the bitmap.


#!/usr/bin/env python

import urllib
import wx
import cStringIO

class Panel1(wx.Panel):
""" class Panel1 creates a panel with an image on it, inherits wx.Panel"""
def __init__(self, parent, id):
wx.Panel.__init__(self, parent, id)
try:
url = "http://www.photoshopsupport.com/photoshop-blog/07/2008/photoshop-blog/istockphoto-free-photo-credits.jpg"
data = urllib.urlopen(url).read()
stream = cStringIO.StringIO(data)
bmp = wx.BitmapFromImage( wx.ImageFromStream( stream ) )
wx.StaticBitmap(self, -1, bmp, (5,5))
except IOError:
print "Image file %s is not found" % url
raise SystemExit

app = wx.PySimpleApp()
frame1 = wx.Frame(None, -1, "An image on a panel", size=(400,300))
Panel1(frame1, -1)
frame1.Show(1)
app.MainLoop()


This is the logger object. It sets up a logger and a formatter as well as a few actual logging methods. These just expose the internal methods of the logging module provided in default python.


#!/usr/bin/env python
import logging
import logging.handlers

class CDLogger(object):

def __init__(self):
self.log = logging.getLogger('cdBurner')
self.log.setLevel(logging.INFO)

hand = logging.FileHandler("cdburner.log", 'a')
form = logging.Formatter("%(asctime)s %(levelname)-10s %(message)s")

hand.setFormatter(form)
self.log.addHandler(hand)

def info(self, message):
self.log.info(message)

def debug(self, message):
self.log.debug(message)

def warning(self, message):
self.log.warning(message)

def error(self, message):
self.log.error(message)



Here is the threading module that is used to execute the timer.


Code on my laptop so I need to get it from the laptop. Will do this later.

Wednesday, April 21, 2010

Amarok Script to Pana Script converter

I use Pana now in favour of Amarok 1.4. Pana is under active development and includes a fix for the cover fetcher that stopped working some time ago.

The next step was to find a way to port the amarok scripts over to Pana. This is easily acheived ( for replay_gain ) anyway with this script:

Source: http://air.elementfx.com/phpBB3/viewtopic.php?f=5&t=3#p3

#! /bin/bash
FILE=$1
FILENAME=`echo ${FILE##*/}`
mkdirerr=$(mktemp)
mkdir $FILENAME".tmp" 2> $mkdirerr
if [ -n "`cat $mkdirerr`" ]; then
cat $mkdirerr
exit
fi
cd $FILENAME".tmp"
echo "Extracting files to $FILENAME.tmp/ ..."
tar jxvf $FILE &> /dev/null

currentdir=`pwd`

for (( i=0 ; $i < `find . -depth -type d | wc -l` ; i=$(( $i+1 )) ));
do
directory[$i]=`find . -depth -type d | head -$(( $i+1 )) | tail -1`
done


for (( i=0 ; $i < ${#directory[@]} ; i=$(( $i+1 )) )); do
cd ${directory[$i]}
for file in `find . -depth -maxdepth 1 -type f ` ; do
echo "Changing "$file
NEWNAME=`echo $file | sed 's/amarok/pana/g'`

sed -i 's/amarok/pana/g' $file
sed -i 's/Amarok/Pana/g' $file

if [ "$file" != "$NEWNAME" ]; then
echo "Renaming file $file to $NEWNAME"
mv $file $NEWNAME
fi
done

cd $currentdir
if [ -z "`echo ${directory[$i]%/*} | grep 'amarok'`" ] && [ "${directory[$i]}" != "." ]; then

NEWDIRNAME=`echo ${directory[$i]##*/}`
NEWDIRNAME=`echo ${NEWDIRNAME} | sed 's/amarok/pana/g'`
NEWDIRNAME=`echo ${NEWDIRNAME} | sed 's/Amarok/Pana/g'`
NEWDIRNAME=${directory[$i]%/*}"/"$NEWDIRNAME
echo "Renaming directory ${directory[$i]} to ${NEWDIRNAME}"
mv ${directory[$i]} ${NEWDIRNAME}
fi
echo "Finished folder ${directory[$i]}"
echo ""
done




NEWFILENAME=$FILENAME
NEWFILENAME=`echo ${NEWFILENAME} | sed 's/amarok/pana/g'`
NEWFILENAME=`echo ${NEWFILENAME} | sed 's/Amarok/Pana/g'`

echo "Recompressing files into $NEWFILENAME"
tar jcvf $NEWFILENAME `ls` &> /dev/null
mv $NEWFILENAME ..
cd ..

echo "rm -f -r ${FILENAME}.tmp/"
# rm -f -r "${FILENAME}.tmp/"


Once you have done all this, then just import the new Pana version of your script using the Pana Script Manager.

Tuesday, March 30, 2010

Howto Install Sun JDK with EE into Ubuntu 10.4

This tutorial is for Ubuntu 10.4 Beta. 10.4 full release is only next month. Well here you go.

  • Download Java for linux from the Oracle Sun website.
  • Download the 32bit version of libstdc++5 from here:
http://packages.debian.org/lenny/amd64/libstdc++5/download
Make sure you get the 32 bit version. It would seem that the installer for Java requires that you have a 32bit version of this library installed. I think this is for the installer GUI only.
  • Extract the library like this:
dpkg-deb -x libstdc++5_3.3.6-18_i386.deb libstdc++5
  • Copy the library into /usr/lib32
sudo cp libstdc++5/usr/lib/libstdc++.so.5.0.7 /usr/lib32/
  • Create a simlink to the file in /usr/lib32/
sudo ln -s libstdc++.so.5.0.7 libstdc++.so.5

Now you can install java like this:
./java_ee_sdk-5_08-jdk-6u18-linux.bin 
All done!

Sunday, March 28, 2010

VirtualBox - AMD-V and KVM

I usually use the KVM on my linux host for Virtualisation. It's default with Fedora and I know it. Usually ...

Today I thought I would try the Ubuntu 10.4 Beta 1 on VirtualBox. I installed the latest VirtualBox ( version 3.1.6 ) and then had a crack at installing the Ubuntu 10.4 64bit. Of course an error came up saying that while AMD-v was enabled, it was not operational. After much googleling about, I found that the KVM modules were causing VirtualBox issues with access.

So a quick "rmmod kvm_amd" and "rmmod kvm" fixed it.

Next time I want to use KVM, I will need to re-enable these with, I suppose, a quick, "insmod kvm_amd" and "insmod kvm"

Wednesday, March 3, 2010

Freeview ( DVB-T ) Linux / Fedora in New Zealand.

I dont have anything fancy other than a fairly good CPU. Its an AMD Athlon(tm) 64 X2 Dual Core Processor 4000+.

I have an avermedia A800 DVB-T tuner from www.nicegear.co.nz

I DO NOT have a VDPAU enabled graphics card which can do on board H.264 decoding for me. That will be soon though. Need to save a bit of cash first.

I am in New Zealand so there are some special areas of concern that needed to be addressed.

Here is a very brief rundown of how i get free view working in my linux.

**** COMMENT ****
Here is a great website. I use this now for post processing.
http://quadpoint.org/projects/simplerip#audio_and_video_bitrates

On my fedora system i have to change -oac copy to -oac pcm and of course I still use my compiled mplayer and the -lavdopts fast:threads=2 option on the 3 lines.

Steps to get Mplayer working.


********************************************************************************
******************** ~./mplayer/channels.conf **********************************
********************************************************************************

TV ONE:538000000:INVERSION_AUTO:BANDWIDTH_8_MHZ:FEC_3_4:FEC_AUTO:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:250+100:300:1200
TV2:538000000:INVERSION_AUTO:BANDWIDTH_8_MHZ:FEC_3_4:FEC_AUTO:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:251+101:301+1010:1201
TVNZ 6:538000000:INVERSION_AUTO:BANDWIDTH_8_MHZ:FEC_3_4:FEC_AUTO:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:252+102:302+1020:1202
TVNZ 7:538000000:INVERSION_AUTO:BANDWIDTH_8_MHZ:FEC_3_4:FEC_AUTO:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:253+103:303+1025:1203
TV3:570000000:INVERSION_AUTO:BANDWIDTH_8_MHZ:FEC_3_4:FEC_AUTO:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:450+300:410:1300
C4:570000000:INVERSION_AUTO:BANDWIDTH_8_MHZ:FEC_3_4:FEC_AUTO:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:451+301:401:1301
TV3 PLUS1:570000000:INVERSION_AUTO:BANDWIDTH_8_MHZ:FEC_3_4:FEC_AUTO:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:452+302:402:1302
Maori Television:666000000:INVERSION_AUTO:BANDWIDTH_8_MHZ:FEC_3_4:FEC_AUTO:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:550+200:600:1400
Parliament TV :666000000:INVERSION_AUTO:BANDWIDTH_8_MHZ:FEC_3_4:FEC_AUTO:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:551+201:601:1401
Test Channel:666000000:INVERSION_AUTO:BANDWIDTH_8_MHZ:FEC_3_4:FEC_AUTO:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:552+202:602:1402
ChineseTV:666000000:INVERSION_AUTO:BANDWIDTH_8_MHZ:FEC_3_4:FEC_AUTO:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:553+203:603:1403
PRIME:666000000:INVERSION_AUTO:BANDWIDTH_8_MHZ:FEC_3_4:FEC_AUTO:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:554+204:604:1404
Reserved 6KSD:666000000:INVERSION_AUTO:BANDWIDTH_8_MHZ:FEC_3_4:FEC_AUTO:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:0:605:1405
Freeview | HD:666000000:INVERSION_AUTO:BANDWIDTH_8_MHZ:FEC_3_4:FEC_AUTO:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:556+206:606:1406
Radio NZ National:666000000:INVERSION_AUTO:BANDWIDTH_8_MHZ:FEC_3_4:FEC_AUTO:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:0+220:620:2000
Radio NZ Concert:666000000:INVERSION_AUTO:BANDWIDTH_8_MHZ:FEC_3_4:FEC_AUTO:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:0+221:621:2001
BaseFM:666000000:INVERSION_AUTO:BANDWIDTH_8_MHZ:FEC_3_4:FEC_AUTO:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:559+209:609:2002
tvCentral:666000000:INVERSION_AUTO:BANDWIDTH_8_MHZ:FEC_3_4:FEC_AUTO:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:560+210:610:1408

********************************************************************************
******************** The DEVEL FILES I INSTALLED ( RELEVANT ONES ) *************
********************************************************************************

Feb 03 21:38:08 Installed: alsa-lib-devel-1.0.22-2.fc12.x86_64
Feb 04 14:32:30 Installed: xorg-x11-drv-nvidia-devel-190.53-1.fc12.x86_64
Feb 04 14:37:13 Installed: gstreamer-devel-0.10.25.1-3.fc12.x86_64
Feb 04 14:37:15 Installed: gstreamer-plugins-base-devel-0.10.25.1-2.fc12.x86_64
Feb 04 14:37:16 Installed: gstreamer-plugins-bad-devel-0.10.17-2.fc12.x86_64
Feb 04 14:37:17 Installed: gstreamer-plugins-good-devel-0.10.17-4.fc12.x86_64
Feb 04 14:42:59 Installed: libvdpau-devel-0.3-1.fc12.x86_64
Feb 04 15:02:07 Installed: x264-devel-0.0.0-0.26.20091026gitec46ace7.fc12.x86_64
Feb 04 15:52:22 Installed: libXvMC-devel-1.0.4-8.fc12.x86_64
Feb 18 17:46:24 Installed: kernel-devel-2.6.31.12-174.2.19.fc12.x86_64
Feb 23 21:53:46 Installed: lame-devel-3.98.2-3.fc11.x86_64
Feb 23 21:53:47 Installed: xvidcore-devel-1.2.1-3.fc12.x86_64


Make sure you also have the relevant libraries that go with these development
packages.


********************************************************************************
******************* COMPILE MPLAYER WITH MULTI THREADING ***********************
********************************************************************************

grab mplayer svn
svn checkout svn://svn.mplayerhq.hu/mplayer/trunk mplayer

now get ffmpeg-mt
git clone git://gitorious.org/ffmpeg/ffmpeg-mt.git

replace the libav* folders in mplayer-svn with the libav* folders in ffmpeg-mt

cd mplayer
./configure
*** FOR 64 BIT FEDORA use ./configure --libdir=/usr/lib64

*** WHEN this completes have a look to see which modules are compiled in. The
*** video out, video codecs, audio out and audio codecs will be listed.
*** If you are missing any you need or think you might need then install the
*** relevant library and -devel packages and run ./configure again.

make

********************************************************************************
******************* RESULTS FROM MY ./configure ********************************
********************************************************************************

Enabled optional drivers:
Input: dvdnav(internal) ftp pvr tv-v4l2 tv-v4l tv libdvdcss(internal) dvdread(internal) vcd dvb network
Codecs: xvid libavcodec(internal) real xanim faad2(internal) libmpeg2(internal) mp3lib(internal) tremor(internal)
Audio output: alsa oss v4l2 mpegpes(dvb)
Video output: v4l2 pnm jpeg matrixview opengl mpegpes(dvb) fbdev xvidix cvidix vdpau xv x11 xover yuv4mpeg md5sum tga

Disabled optional drivers:
Input: vstream radio tv-dshow live555 nemesi cddb cdda smb
Codecs: libschroedinger libdirac x264 libdv libopencore_amrwb libopencore_amrnb qtx win32 faac musepack libdca liba52 libtheora speex toolame twolame libmad liblzo gif OpenJPEG
Audio output: sun openal jack pulse nas esd arts ivtv dxr2 sdl
Video output: zr zr2 ivtv dxr3 dxr2 sdl vesa gif89a svga caca aa ggi xmga mga winvidix 3dfx dga xvmc dfbmga directfb bl xvr100 tdfx_vid wii s3fb tdfxfb

********************************************************************************
******************* Here is a link to a person who did it in mandriva **********
********************************************************************************
http://blog.linuxbox.co.nz/2008/08/hauppauge-wintv-hvr900-on-mandriva.html

********************************************************************************
******************* Various mplayer commands ***********************************
********************************************************************************

RECORD A CHANNEL INDEFINATELY ( KILL OR CTRL+C TO END THE THING )
#!/bin/bash
/home/dave/devel/mplayer-mt/mplayer_svn/mplayer -dvbin file=/home/dave/.mplayer/channels.conf -dumpfile /home/dave/Videos/$1.ts -dumpstream dvb://"$2"


ENCODE A RECORDING
#!/bin/bash
/home/dave/devel/mplayer-mt/mplayer_svn/mencoder -oac lavc -ovc lavc -of mpeg -mpegopts format=xsvcd -vf scale=480:576,harddup -srate 44100 -af lavcresample=44100 -lavcopts vcodec=mpeg2video:mbd=2:keyint=15:vrc_buf_size=917:vrc_minrate=600:vbitrate=2500:vrc_maxrate=2500:acodec=mp2:abitrate=224:aspect=16/9 -ofps 25 -lavdopts fast:threads=4 -mc 10 -o /data/$1.mpg -quiet $1


WATCH A RECORDING. THIS IS THE RAW STREAM CAPTURED DURING BY THE RECORD A CHANNEL SCRIPT ABOVE.

#!/bin/bash
/home/dave/devel/mplayer-mt/mplayer_svn/mplayer -vc ffh264 -vo gl -lavdopts fast:threads=4 -vf pp=fd -mc 10 -framedrop -cache 10240 -delay -0.3 /home/dave/Videos/$1


WATCH TV STRAIGHT. THIS IS FINE FOR ALL CHANNELS EXCEPT TV3 WHICH IS AT FULL HD AND MY CPU DOESNT DECODE IT FAST ENOUGH. EVEN IN MULTI THREADING.

#!/bin/bash
/home/dave/devel/mplayer-mt/mplayer_svn/mplayer -vo gl -lavdopts fast:threads=2 -mc 10 -framedrop -vf pp=fd -cache 10240 -delay -0.3 dvb://"$1"

So have fun.

Friday, January 8, 2010

chomping isql and date conversion

Today a colleague was working on retrieving data from a Windows SQL Server database from within linux. He needed a bash script that would execute a query against the database and dump the results in comma delimited format.

The SQL query he wrote was rather long and so was broken up into new lines and indented to ease editing and readability. This, however, caused a snag in isql.

isql is a commandline tool that ships with linux ODBC that can be used to connect to an ODBC DSN and execute sql commands as well as execute batch files in unattended mode.

isql parses batch files ( essentially SQL scripts ) in such a way as that each line is treated as a single command. It does not recognise the GO statement or any other command terminator.

Our workaround was to chomp the script prior to piping it into isql. So here we go:

Here is an example script.

#--- mySqlScript.sql ---

SELECT TOP 100 *
FROM myTable t
WHERE t.created_at between '2009-01-01' and '2009-12-31'
AND t.enabled = 1


As you can tell the script is formatted in a nice and legible way which would break in isql batchmode.

The fix is rather simple but also elegant because we can keep the script in it's original format and remove the newlines ( chomp ) just before it is sent to isql.

Here is the commandline that does just that.

[dave@fedora junk]$ cat mySqlScript.sql | perl -ne 'chomp; print unless /^$/'
SELECT TOP 100 *FROM myTable tWHERE t.created_at between '2009-01-01' and '2009-12-31' AND t.enabled = 1[dave@fedora junk]$


So the final step is to pipe all that into an isql command.

[dave@fedora junk]$ cat mySqlScript.sql | perl -ne 'chomp; print unless /^$/' | isql -b -d, my_dsn userName passW0rd


Now the above script failed on my computer because (a) I do not have linux odbc installed at home and (b) I do not have a dsn called my_dsn to connect to.
The chomping, however, does work!