Category Archives: Source

Hadoop1.2.1分散型クラスタ環境でpig0.12をインストール

インストールと設定
1.pig-0.12.0.tar.gzをダウンロード
tar -zxvf pig-0.12.0.tar.gz
sudo mv pig-0.12.0 /opt/pig-0.12.0

2.環境変数を設定
sudo gedit /etc/profile
ファイルの最後に下記を追加
export PIG_HOME=/opt/pig-0.12.0
export PIG_CLASSPATH=${HADOOP_HOME}/conf
export PATH=${PATH}:${PIG_HOME}/bin

source /etc/profile
3.検証と実行
3.1 ローカルで実行
コマンド: pig -x local
下記メッセージが出れば、ローカルでうまく動いている
warning: $HADOOP_HOME is deprecated
2014-07-09 12:07:43,713 「main」 INFO org.apache.pig.main -Apache pig version
2014-07-09 12:07:43,714「main」 INFO org.apache.pig.main -Logging error message
2014-07-09 12:07:43,790 「main」 INFO org.apache.pig.main.impl.util.utils-
3.3 クラスタ実行
コマンド: start-all.sh //hadoopクラスタが起動
jps //起動の成功かどうかを確認
または pigでクラスタ実行

IDEA MavenプロジェクトにJettyプラグインを追加する方法

<build>に下記内容を追加:

<build>
<finalName>MavenDemoTest</finalName>
<plugins>
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>maven-jetty-plugin</artifactId>
<version>6.1.26</version>
<configuration>
<connectors>
<connector implementation=”org.mortbay.jetty.nio.SelectChannelConnector”>
<port>8080</port>
</connector>
</connectors>
<scanIntervalSeconds>10</scanIntervalSeconds>
</configuration>
</plugin>
</plugins>
</build>

WindowsにPython easy_installをインストール

WindowsにPython easy_installをインストール
1.ダウンロードURL:
https://pypi.python.org/pypi/setuptools

2.注意事項:

32ビットのwindows7にsetuptools-0.6c11.win32-py2.7.exeをダウンロード

64ビットのwindows7 にez_setup.exeをダウンロードすることが必要です

3.インストール方法
3.1 ez_setup.pyをダウンロード
3.2 起動->cmd->python ez_setup.pyを実行,setuptoolsをインストール
3.3 C:\Program Files\python2.7\scripts直下にeasy_installをインストール

4.システムの環境変数を設定
プロパティ=>システムの詳細設定=>環境変数=>システム環境変数=>新規
新しいシステム変数に下記を設定する
変数名(N):Path
変数値(V):D:\Program Files\python2.7\scripts

sqlmap基本コマンドの使い方

基本コマンド:

./sqlmap.py –h //ヘルプの情報の確認

./sqlmap.py –u “XXX” //get

./sqlmap.py –u “XXX” –data “DATA”//post

./sqlmap.py –u “XXX” –cookie “COOKIE”//リクエストcooikeを修正

./sqlmap.py –u “XXX” –dbs //データベース列

./sqlmap.py –u “XXX” –-users //ユーザー

./sqlmap.py –u “XXX” –-passwords //パスワードを取得ハッシュ

./sqlmap.py –u “XXX” –-tables -D DB_NAME //DB_NAMEのテーブル列

./sqlmap.py –u “XXX” –-columns –T TB_NAME -D DB_NAME //TB_NAMEの列を読み取る

./sqlmap.py –u “XXX” –-dump –C C1,C2,C3 –T TB_NAME -D DB_NAME //フィールドC1、C2、C3データを読み取る

./sqlmap.py –u “XXX” –-os-shell //shellを取得

Ubuntu 12.04にPwn Pad 2014をインストールする方法

1.Ubuntuに”adb”と”fastboot”をインストール
$ sudo add-apt-repository ppa:nilarimogard/webupd8
$ sudo apt- get update
$ sudo apt- get install android-tools-adb android-fastboot-tools

2.Pwn Pad 2014システムファイルを解凍する
ダウンロードURL:
https://www.pwnieexpress.com/support/downloads/

$ tar xvf pwnpad-2014_image.tar.xz
$ cd pwnie_img/ 3
$ chmod +x imagev2.sh

3.Nexus 7はPCに接続してfastbootモードに入る

4.adaを初期化する
$ sudo adb start-server

5.Pwn Padの実装スクリプトを実行
$ sudo ./imagev2.sh

6.画面の説明ヒントに従って操作
Pree ENTER to continue,CTR+C to abort
Boot the device into fastboot mode (hold power and volume down)
Attach the device vis USB once in fastboot mode.
Press 「Enter」key to continue…

 

WindowsとLinux環境にAndroidでapk署名finggerprintを取得方法

システム要件:
JDKをインストール済み

1.apkのrsaファイルを検索
Windows:
> jar tf HelloWorld.apk |findstr RSA
Linux:
$ jar tf HelloWorld.apk |grep RSA
META-INF/CERT.RSA

2.apkからrsaファイルを解凍
jar xf HelloWorld.apk META-INF/CERT.RSA

3.署名のfingerprintsを取得
keytool -printcert -file META-INF/CERT.RSA


Certificate fingerprints:
MD5: BC:6D:BD:6E:49:69:2A:57:A8:B8:28:89:04:3B:93:A8
SHA1: 0D:DF:76:F4:85:96:DF:17:C2:68:1D:3D:FF:9B:0F:D2:A1:CF:14:60
Signature algorithm name: SHA1withRSA
Version: 3

4.クリーンアップ作業、rsaファイルを削除
Windows
rmdir /S /Q META-INF

Linux
rm -rf META-INF

CentOS起動時にTomcat自動起動のプログラムを設定する方法

vim /etc/rc.d/rc.local

# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don’t
# want to do the full Sys V style init stuff.
export JAVA_HOME=/home/hostname/jdk1.6.0_31
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin
export CATALINA_HOME=/home/hostname/apache-tomcat-6.0.35
#tomcatが自動起動
/home/hostname/apache-tomcat-6.0.35/bin/startup.sh
#他のプログラムの起動サンプル
/usr/bin/
touch /var/lock/subsys/local

Windows上にhadoop2.2.0を配置する方法

1.core-site.xmlを編集
<?xml version=”1.0″ encoding=”UTF-8″?>
<?xml-stylesheet type=”text/xsl” href=”configuration.xsl”?>
<!–
Licensed under the Apache License, Version 2.0 (the “License”);
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an “AS IS” BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
–>

<!– Put site-specific property overrides in this file. –>

<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>

2.hdfs-site.xmlを編集
<?xml version=”1.0″ encoding=”UTF-8″?>
<?xml-stylesheet type=”text/xsl” href=”configuration.xsl”?>
<!–
Licensed under the Apache License, Version 2.0 (the “License”);
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an “AS IS” BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
–>

<!– Put site-specific property overrides in this file. –>

<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/hadoop/data/dfs/namenode</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/hadoop/data/dfs/datanode</value>
</property>
</configuration>

3.yarn-site.xmlを修正
<?xml version=”1.0″?>
<!–
Licensed under the Apache License, Version 2.0 (the “License”);
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an “AS IS” BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
–>
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
</configuration>

4.mapred-site.xmlを修正
<?xml version=”1.0″?>
<?xml-stylesheet type=”text/xsl” href=”configuration.xsl”?>
<!–
Licensed under the Apache License, Version 2.0 (the “License”);
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an “AS IS” BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
–>

<!– Put site-specific property overrides in this file. –>

<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>

5.起動->cmd 「hadoop namenode -format」を実行
実行結果は以下の通りです。
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

C:\Users\abhijitg>cd c:\hadoop\bin

c:\hadoop\bin>hdfs namenode -format
14/11/03 18:07:47 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = ABHIJITG/x.x.x.x
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 2.2.0
STARTUP_MSG: classpath = <classpath jars here>
STARTUP_MSG: build = Unknown -r Unknown; compiled by ABHIJITG on 2014-11-01T14:42Z
STARTUP_MSG: java = 1.7.0_03
************************************************************/
Formatting using clusterid: CID-1af0bd9f-efee-4d4e-9f03-a0032c22e5eb
14/11/03 18:07:48 INFO namenode.HostFileManager: read includes:
HostSet(
)
14/11/03 18:07:48 INFO namenode.HostFileManager: read excludes:
HostSet(
)
14/11/03 18:07:48 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit=1000
14/11/03 18:07:48 INFO util.GSet: Computing capacity for map BlocksMap
14/11/03 18:07:48 INFO util.GSet: VM type = 64-bit
14/11/03 18:07:48 INFO util.GSet: 2.0% max memory = 888.9 MB
14/11/03 18:07:48 INFO util.GSet: capacity = 2^21 = 2097152 entries
14/11/03 18:07:48 INFO blockmanagement.BlockManager: dfs.block.access.token.enable=false
14/11/03 18:07:48 INFO blockmanagement.BlockManager: defaultReplication = 1
14/11/03 18:07:48 INFO blockmanagement.BlockManager: maxReplication = 512
14/11/03 18:07:48 INFO blockmanagement.BlockManager: minReplication = 1
14/11/03 18:07:48 INFO blockmanagement.BlockManager: maxReplicationStreams = 2
14/11/03 18:07:48 INFO blockmanagement.BlockManager: shouldCheckForEnoughRacks = false
14/11/03 18:07:48 INFO blockmanagement.BlockManager: replicationRecheckInterval = 3000
14/11/03 18:07:48 INFO blockmanagement.BlockManager: encryptDataTransfer = false
14/11/03 18:07:48 INFO namenode.FSNamesystem: fsOwner = ABHIJITG (auth:SIMPLE)
14/11/03 18:07:48 INFO namenode.FSNamesystem: supergroup = supergroup
14/11/03 18:07:48 INFO namenode.FSNamesystem: isPermissionEnabled = true
14/11/03 18:07:48 INFO namenode.FSNamesystem: HA Enabled: false
14/11/03 18:07:48 INFO namenode.FSNamesystem: Append Enabled: true
14/11/03 18:07:49 INFO util.GSet: Computing capacity for map INodeMap
14/11/03 18:07:49 INFO util.GSet: VM type = 64-bit
14/11/03 18:07:49 INFO util.GSet: 1.0% max memory = 888.9 MB
14/11/03 18:07:49 INFO util.GSet: capacity = 2^20 = 1048576 entries
14/11/03 18:07:49 INFO namenode.NameNode: Caching file names occuring more than 10 times
14/11/03 18:07:49 INFO namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
14/11/03 18:07:49 INFO namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 0
14/11/03 18:07:49 INFO namenode.FSNamesystem: dfs.namenode.safemode.extension = 30000
14/11/03 18:07:49 INFO namenode.FSNamesystem: Retry cache on namenode is enabled
14/11/03 18:07:49 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time
is 600000 millis
14/11/03 18:07:49 INFO util.GSet: Computing capacity for map Namenode Retry Cache
14/11/03 18:07:49 INFO util.GSet: VM type = 64-bit
14/11/03 18:07:49 INFO util.GSet: 0.029999999329447746% max memory = 888.9 MB
14/11/03 18:07:49 INFO util.GSet: capacity = 2^15 = 32768 entries
14/11/03 18:07:49 INFO common.Storage: Storage directory \hadoop\data\dfs\namenode has been successfully formatted.
14/11/03 18:07:49 INFO namenode.FSImage: Saving image file \hadoop\data\dfs\namenode\current\fsimage.ckpt_00000000000000
00000 using no compression
14/11/03 18:07:49 INFO namenode.FSImage: Image file \hadoop\data\dfs\namenode\current\fsimage.ckpt_0000000000000000000 o
f size 200 bytes saved in 0 seconds.
14/11/03 18:07:49 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
14/11/03 18:07:49 INFO util.ExitUtil: Exiting with status 0
14/11/03 18:07:49 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at ABHIJITG/x.x.x.x
************************************************************/

mamp proのmysqlコマンド場所

場所:

/Applications/MAMP/Library/bin

CentOS 7に ifconfig command not found の解決方法

エラーメッセージ:
# ifcon
-bash: ifconfig: command not found

1.ifconfigで「/sbin/ifconfig」存在かどうか確認
2.ifconfigコマンドが存在するなら、環境変数の設定を確認する
#echo $PATH

環境変数にifconfigコマンドを含まれない場合
対策:
2.1 一時的な環境変数を変更:シェルで入力
$export PATH = $PATH:/sbin
再度ifconfigを入力
2.2 永久に「/sbin」パスが含まれるようにPATH変数を変更
vim /etc/profile
「export PATH=$PATH:/sbin」を入力
保存してXが起動

3.ifconfigが存在しない場合
yum upgrade
yum install net-tools