Technical Tidbits
A collection of code snippets of sometimes rarely used, or perhaps difficult to remember tasks, commands and code snippets.
Changing a YAML file in Ruby
require "yaml"
File.open("./example.yml", 'w') {|f| f.write("
workbook: 5
sheet: 10
header_index: 5
headers:
- hello
- hi
- bye
") }
data = YAML.load_file("./example.yml")
puts "OLD DATA: #{data}"
data["headers"] = [ "Hello", "Hi", "Bye" ]
File.open("./example.yml", 'w') {|f| f.write(data.to_yaml) }
data = YAML.load_file("./example.yml")
puts "NEW DATA: #{data}"
Clean bundler within a Ruby Script
oenv = ENV.to_hash
begin
%w{BUNDLE_GEMFILE RUBYOPT GEM_HOME GIT_DIR GIT_WORK_TREE BUNDLE_BIN_PATH}.each { |key| ENV.delete(key) }
output = IO.popen("xlshell "#{filename}" "#{worksheet}"")
return output.readlines.collect { |f| f.strip }
ensure
ENV.replace(oenv)
end
Clear Your Local DNS Cache
dscacheutil -flushcache
On newer Macs, somethings this works too.
sudo killall -HUP mDNSResponder
Create a public / private key
ssh-keygen -t rsa
Dealing with CSV files and MS Excel (.xls / .xlsx)
When dealing with CSV, you need to get your encodings correct.Excel will typically export to ISO-8859-1. This is not good, and should be converted to UTF-8 to support non english files.
To check the encoding of a file
file -I original_file.csv
To convert between encodings (a few common examples)
iconv --from-code=ISO-8859-1 --to-code=UTF-8 original_file.csv > new_file.utf8.csv
iconv --from-code=MacRoman --to-code=UTF-8 original_file.csv > new_file.utf8.csv
When exporting files to be read within Excel, you will need to convert the CSV based on the target platform.
For MAC
iconv --from-code=UTF-8 --to-code=MacRoman downloaded_file.csv > downloaded_file_mac.excel.csv
For PC (untested)
iconv --from-code=UTF-8 --to-code=Windows-1252 downloaded_file.csv > downloaded_file_windows.excel.csv
Deleting Git Branches
Delete the "demo_maintenance" branch on the "origin" remote server.
git push origin :demo_maintenance # deletes the remote branch
git branch -d demo_maintenance # deletes the local branch
If you need to 'force' the delete on your local machine, use -D
git branch -D demo_maintenance # deletes the local branch
Delete Rails sessions
Remove all old sessions, and then optimize the table.</p>
DELETE
FROM sessions
WHERE updated_at < DATE_SUB(NOW(), INTERVAL 1 DAY);
optimize TABLE sessions;
Destroy All RabbitMQ Queues
Should only be done on a dev machine:
rabbitmqctl stop_app
rabbitmqctl reset
rabbitmqctl start_app
Faster Macbook Pro Wakeups
A longer discussion at ewal.net
sudo pmset -a standbydelay 86400 # only standby after 24 hours
Find text in files within a directory
grep -r "some words reward" /path/tp/look/*
Find large files on a Linux / Mac OS X machine
The following should owrk on your Mac OS X
sudo find / -type f -size +20000k -exec ls -lh {} ; | awk '{ print $9 ": " $5 }'
For a linux (tested on Ubuntu), the $9 most liley needs to be an $8
sudo find / -type f -size +20000k -exec ls -lh {} ; | awk '{ print $8 ": " $5 }'
Enable FTP on Mac OS X Lion
# enable the ftp server
sudo -s launchctl load -w /System/Library/LaunchDaemons/ftp.plist
#disable the server when you are done
sudo -s launchctl unload -w /System/Library/LaunchDaemons/ftp.plist
Checking Out New Git Branches
First locat the branches, then checkout the one you want (e.g. origin/demo1)
git branch -r
git checkout --track -b demo1 origin/demo1
Undo a local commit
If you committing a git change locally, you can undo it using the following command:
git reset --soft HEAD^
Undo the last pushed commited
If you already committed the change, then you need to do hard resets:
git reset --hard HEAD~1
git push -f
Granting passwordless access to root on RHEL
When running CHEF, it needs root access, so if you are using something like chef-solo, and you like automated scripts, then here is how grant that access. PLEASE NOTE, THIS IS VERY UNSAFE, to avoid try, I created a (now obsolete) CHEF BOOTSTRAP.
$ mkdir -p /root/.ssh
$ chmod 700 /root/.ssh
$ chmod 600 /roiot/.ssh/authorized_keys
$ restorecon -R -v /root/.ssh
Install older version of HTML to PDF (wkhtmltopdf) using Homebrew on a Mac
Bug with current version that will cause the process to not exit properly:</p>
brew uninstall wkhtmltopdf
cd /usr/local/Library/Formula/
git checkout 6e2d550 /usr/local/Library/Formula/wkhtmltopdf.rb
brew install wkhtmltopdf
wkhtmltopdf --version | grep 0.9.9
Ticket was described here, but now the site (and history) have for wkhtmltopdf.org have moved. Thank you to wearepandr
Kill a Defunct Process
# Locate defunct processes
ps -A | grep defunct
# Get the parent ID (UID PID PPID ...)
ps -ef | grep defunct | more
# for example...
deployer 2707 2698 0 2012 ? 00:00:03 [unicorn_rails] <defunct>
deployer 2710 2698 0 2012 ? 00:00:04 [unicorn_rails] <defunct>
# Kill the parent PID (PPID)
kill -9 2698
See how much memory something consumes in PHP
$start_time = microtime(true);
$start_memory = memory_get_usage();
// INSERT CODE HERE
$end_memory = memory_get_usage();
$end_time = microtime(true);
print_r(array(
'memory (Mb)' => ($end_memory - $start_memory) / (1024 * 1024),
'time (s)' => $end_time - $start_time
));
Memcache in PHP
$memcache = new Memcache();
$memcache->connect('localhost', 11211);
function doSomethingSlow() {
sleep(5);
return "foobar";
}
$name = $memcache->get('name');
if (!$name)
{
$name = doSomethingSlow();
$memcache->set('name',$name);
}
echo $name;
Execute a single MySQL statement from command line
mysql -u myname -pmypassword -D mydb -e "show tables"
Granting access to mysql users
SHOW GRANTS FOR deployer
# ALL ACCESS
GRANT ALL ON myshop.* TO 'deployer'@'%' IDENTIFIED BY 'pass' ;
# SELECT ONLY, and filtered by IP
GRANT SELECT ON myshop.* TO deployer@'192.168.1.%' IDENTIFIED BY 'pass';
# Or by domain name
GRANT SELECT ON myshop.* TO deployer@'webapp.myserver.com' IDENTIFIED BY 'pass';
FLUSH PRIVILEGES;
Removing access to mysql users
REVOKE ALL PRIVILEGES, GRANT OPTION FROM 'deployer'@'%';
How to change owners of a postgres database
GRANT ALL PRIVILEGES ON DATABASE @DATABASE_NAME@ TO @NEW_USERNAME@;
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO @NEW_USERNAME@;
For example, changing from the 'postgres' user to the 'deployer' user.
GRANT ALL PRIVILEGES ON DATABASE my_app TO deployer;
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO deployer;
Granting access to postgres users
CREATE USER myuser WITH ENCRYPTED PASSWORD 'ppp';
GRANT USAGE ON SCHEMA public to myuser;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO myuser;
GRANT CONNECT ON DATABASE mydb to myuser;
c mydb
GRANT USAGE ON SCHEMA public to myuser;
GRANT SELECT ON ALL SEQUENCES IN SCHEMA public TO myuser;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO myuser;
Clean / Clear / Delete Rails Databse Sessions
Don't forget to clear your sessions every once in a while
rake db:sessions:clear
Run A Script Remotely Using SSH
For simple scripts, just pass them via a quoted string.
ssh name@server.com 'ls -la'
For more complicated scripts, put them in a local file and stream them to the server.
ssh name@server.com 'bash -s' < local_script.sh
Rename exentions on a file
find . -name '*.php.eex' -exec sh -c 'mv "$0" "${0%.php.eex}.eex"' {} ;
find . -name '*.php' -exec sh -c 'mv "$0" "${0%.php}.eex"' {} ;
Maintain should support in latest rspec gem
If you receive warnings about should and either are not ready to change, or don't want to, here is how you can silence the warnings.
Using should
from rspec-expectations' old :should
syntax without explicitly enabling the syntax is deprecated. Use the new :expect
syntax or explicitly enable :should
instead.
Using stub
from rspec-mocks' old :should
syntax without explicitly enabling the syntax is deprecated. Use the new :expect
syntax or explicitly enable :should
instead.
RSpec.configure do |config|
config.mock_with :rspec do |c|
c.syntax = [:should, :expect]
end
config.expect_with :rspec do |c|
c.syntax = [:should, :expect]
end
end
Tar / Untar a file
Compress a directly into a tar.gz file:
tar cvzf myapp.tar.gz myapp/
Uncompress that file back into a directory:
tar zxfv myapp.tar.gz
Uncompress a gz zip (no tar):
gzcat x.txt.gz >x.txt
gunzip -c x.txt.gz >x.txt
Which Port is a Process Running On
Find which process is listening on a certain port
lsof -i :12345 # for port 12345