Showing posts with label mySQL. Show all posts
Showing posts with label mySQL. Show all posts

Saturday, February 16, 2013

MDL: Some More XBMC Tweaks

I’m now running two instances of XBMC Frodo each sharing a single MySQL database.  Frodo has allowed me to strip out some of the bandaids I put in place to make the experience as seamless as possible.
The box that I have in my family room is an Acer Aspire running Ubuntu.  That is also where MySQL is running.  The box I have in the basement is a Raspberry Pi running RaspBMC.
I used to have to store all of my thumbnails on an NFS mount and create linked directories on every box because XBMC wouldn’t regenerate thumbs locally if the content was already defined in the database.  Now with Frodo that isn’t necessary as XBMC will create thumbnails locally regardless of the existence in the database.
The other piece that I was able to strip out was a kludge of a script that attempted to force regeneration of  metadata and thumbnails when I made changes after the initial discovery.  MediaElch now allows me to edit metadata and force XBMC discovery of those change.
My tweaking is now limited to:
  • Creating symlinked NFS mounts for my content so that they show up in local paths (this is a work around so that I can use both Windows and Linux as XBMC hosts).
  • Some keymap customizations to work around limited remote functionality using Raspberry Pi and CEC.
  • Creating some dependencies for MySQL and XBMC.
Creating the database dependency in my family room was easy since the database was local.  I just needed to edit /etc/init/xbmc-live.conf and add:
start on started mysql 
pre-start script
while [ `mysqladmin ping -h 192.168.1.130 -uxbmc -pxbmc -s | grep -c alive` -eq$
do
  sleep 1
done
end script
sleep 5

Creating the dependency on the Pi was a bit more challenging.  After installing mysql-client via apt-get I added this code to the top of /scripts/xbmc-watchdog.sh.
#check to see if mySQL is running on the family room PC
while [ `mysqladmin ping -h 192.168.1.130 -uxbmc -pxbmc -s | grep -c alive` -eq 0 ]
do
  sleep 1
done

sleep 5

I’m reasonable sure that storing your password cleartext in a configuration script is dumb, but I doubt anyone will be trying to break in and sabotage my music collection.  I also don’t know if this is precisely the correct place to put the delay – but it seems to work so I’m declaring success.

edited to add pre-start script to family room config

Saturday, April 14, 2012

MDL: Maintain the XBMC Library

One of the most frustrating things about XBMC is the static nature of its library.  The forums are full of people asking “How do you force the XBMC library to refresh?”  Many of them are asking specifically about the thumbnails and fanart.  My biggest concern is the metadata.

I have a mild case of OCD and am constantly tweaking the information about my movie collection.  Unfortunately, XBMC doesn’t ever pick up this information.  You can refresh movies manually, one by one.  But that process is tedious at best.

You can also delete the database and rebuild it.  But that kind of defeats the purpose of having a library, doesn’t it?  If you go this path you lose watch history, ratings and other information.

I think that I have developed a script that accomplishes what I need.  It removes just enough information to force re-population without getting rid of the things that I care about.  I currently have it setup to run once per week, which should be more than sufficient for my needs.

The script is in PowerShell, but shouldn’t be overly difficult to rewrite in just about anything.  A couple notes:

a) I am not worried about refreshing pictures here.  Though that can be accomplished pretty easily by deleting the thumbnail cache.

b) Use at your own risk.  I am not 100% certain that this is stable in all circumstances.  I’ve run it a bunch on my setup, but make no guarantees what it will do to yours.

c) I’m pretty sure its inefficient, I am not a programmer by nature.  If you find methods to improve it, please do so and post your results back here.

# You should be able to edit here and the rest of the script should work
# With the very important exception of the section marked !!!!LOOK AT ME!!!! below.
$xbmcuserid = "user"
$xbmcpassword = "pass"
$xbmcIP = "192.168.1.2"
$xbmcPort = "8000"
$mySQLIP = "192.168.1.2"
$url = "http://" + $xbmcIP + ":" + $xbmcPort + "/jsonrpc"
$xbmcDB = "xbmc_video"

# Connect to the mySQL database
[void][System.Reflection.Assembly]::LoadWithPartialName("MySql.Data")
$XBMCConnection = New-Object MySql.Data.MySqlClient.MySqlConnection
$XBMCConnection.ConnectionString = "database=" + $xbmcDB + ";server=" + $mySQLIP + ";Persist Security Info=false;user id=" + $xbmcuserid + ";pwd=" + $xbmcpassword
$XBMCConnection.Open()

# Make a copy of the movie table
$copyCommand = $XBMCConnection.CreateCommand()
$copyCommand.CommandText = "CREATE TABLE movie_copy SELECT * FROM movie"
$copyCommand.ExecuteNonQuery()

# Clean out the movie table
$cleanCommand = $XBMCConnection.CreateCommand()
$cleanCommand.CommandText = "DELETE FROM movie WHERE idMovie > 0"
$cleanCommand.ExecuteNonQuery()

# !!!!!!!!!!!!!!LOOK AT ME!!!!!!!!!!!!!!
# You are going to need to clean out the hashes associated with the movies or the scan will just skip past everything
# You can clear out all of the hashes (movies and TVs) but that will cause everything to rescan
# I prefer to just scan the stuff that I care about YMMV
# !!!!!!!!!!!!!!LOOK AT ME!!!!!!!!!!!!!!
$cleanHashCommand = $XBMCConnection.CreateCommand()
$cleanHashCommand.CommandText = "UPDATE path` SET `strHash` = '0' WHERE strPath like '%Movies/%';"
$cleanHashCommand.ExecuteNonQuery()

# Ask XBMC to look for new videos - this will reload the NFOs into the database
$command = '{"jsonrpc": "2.0", "method": "VideoLibrary.Scan", "id": 1}'
$bytes = [System.Text.Encoding]::ASCII.GetBytes($command)
$web = [System.Net.WebRequest]::Create($url)
$web.Method = "POST"
$web.ContentLength = $bytes.Length
$web.ContentType = "application/x-www-form-urlencoded"
$stream = $web.GetRequestStream()
$stream.Write($bytes,0,$bytes.Length)
$stream.close()
$reader = New-Object System.IO.Streamreader -ArgumentList $web.GetResponse().GetResponseStream()
$reader.ReadToEnd()
$reader.Close()

# Wait for 30 minutes, this is probably me being lazy. 
# I can't think of a way to see if the scanning has been completed or not
start-sleep -s 1800

# Reset the movie ids back to what they were. 
# This is required if you a) care about Recent Movies being accurate.  b) you use sets c) probably something else too
$ResetIDCommand = $XBMCConnection.CreateCommand()
$ResetIDCommand.CommandText = "UPDATE " + $xbmcDB + ".movie_copy t1, " + $xbmcDB + ".movie t2 SET t2.idMovie=t1.idMovie WHERE t1.idFile = t2.idFile"
$ResetIDCommand.ExecuteNonQuery()

# Drop the backed up table
$DropCommand = $XBMCConnection.CreateCommand()
$DropCommand.CommandText = "DROP TABLE " + $xbmcDB + ".movie_copy"
$DropCommand.ExecuteNonQuery()

# Clean the database
# Don't know if this is necessary, but it can't hurt, right?
$command = '{"jsonrpc": "2.0", "method": "VideoLibrary.Clean", "id": 1}'
$bytes = [System.Text.Encoding]::ASCII.GetBytes($command)
$web = [System.Net.WebRequest]::Create($url)
$web.Method = "POST"
$web.ContentLength = $bytes.Length
$web.ContentType = "application/x-www-form-urlencoded"
$stream = $web.GetRequestStream()
$stream.Write($bytes,0,$bytes.Length)
$stream.close()
$reader = New-Object System.IO.Streamreader -ArgumentList $web.GetResponse().GetResponseStream()
$reader.ReadToEnd()
$reader.Close()

Sunday, August 14, 2011

MDL: Optimizing XBMC

There are a number of tweaks that I had to make on XBMC to make it work a little better with my setup.

First – since I am using MySQL I had to add some indexes and make a small change to the MySQL server.

By default it appears that MySQL wants to do a reverse DNS lookup to all client connections.  This makes the initial connection very slow, to disable this behavior add the following to my.ini in the [mysqld] section:

skip-name-resolve

The following indexes also improve performance:

use XBMC_video;
ALTER TABLE movie ADD INDEX idMovie(idMovie);
ALTER TABLE movie ADD INDEX idFile(idFile);

Another change that is essential because of the shared MySQL database is the sharing of the thumbnail directory – otherwise the thumbnails will only appear on the XBMC client that discovers the media.  To get around this I created a symbolic link on the Linux box to the shared Thumbnail directory on Windows.

Connecting to Windows shares from Linux has caused me all sorts of issues.  I initially tried mounting shares in fstab, but if the Windows box wasn’t available during Linux boot the shares were unavailable.  I switched to automounter which resolved the issue.

There are some other SAMBA tweaks to improve performance – in theory this was supposed to prevent video from stopping if it was paused or looking at info.  So far it is only partially successful.

socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192

Finally, ever time I upgrade something in XBMC or a supporting component the autostart fails to actually, you know, autostart XBMC.  To fix this edit the /etc/uxlaunch/uxlaunch file with the correct home directory.

Technorati Tags: ,,

Sunday, June 26, 2011

MDL: Backing Up to Flickr Part I

For better or worse I’ve decided that I would use flickr as my photo backup solution.  For one, it’s not terribly expensive – around $30 for unlimited storage – and it makes picture sharing pretty easy.  There are privacy concerns for some, but I’m simply not sure that I care.
The downside is that there aren’t really any tools out there that automate sending the pictures to flickr.  Foldr Monitr comes close, but its buggy, isn’t updated anymore and I just don’t like the choices that it has made. 
After about two years of complaining I guess I’ll just have to handle the problem myself.  Odds are that you won’t like the choices that I’ve made, but hopefully I’ve given you enough pieces that you can make the necessary modifications without having to start from scratch the way that I did.
You are going to need a database of some kind to keep track of everything – I have a copy of MySQL running already for XBMC so that’s what I’ll use.  I’m also going to use PowerShell since that is what I know which means that I’ll have to install the ODBC driver for MySQL.
Finally, I suck as a programmer.  Don’t use any of this code with the expectations that you’ll learn how to get better at scripting.  Also, don’t point out how if I changed some bit of code around it would be SOOO MUCH BETTER!   I don’t care – if my code functions for me, inefficient or not, I am quite content.
So be warned – this thing works for me and my needs 95% of the time.  And if I run across an anomaly where it doesn’t do what is expected if x & y are true – I stop doing  both x & y at the same time.  The chances that I’ll update the script to accommodate your special circumstances are pretty damn rare.  And adding any functionality at all is all but unthinkable.
All that being said, feel free to borrow the code and adapt it to your own purposes – you just can’t take this and put it into something that you are going to sell.  And if you update the scripts for public consumption – please give me my due by linking back here.  Obviously while noting how crappy the code is and how much work you had to put into it to make it semi-functional for a normal human being.
Part II will get into the script and perhaps the database – assuming that I actually finishing writing the god damned thing.

Update: I have completely abandoned this project.  Flickr is just too much of a pain in the ass.  I am almost completely migrated to CrashPlan for offsite backups of my pictures with the plus of being able to backup non-pictures as well.

Technorati Tags: ,,,