免费注册 查看新帖 |

Chinaunix

  平台 论坛 博客 文库
最近访问板块 发新帖
查看: 2909 | 回复: 0

puppet下有个puppet-load.rb的测试性能的脚本,但是我执行报错。没办法来ruby这里看看 [复制链接]

论坛徽章:
0
发表于 2013-05-03 18:39 |显示全部楼层
  1. #!/usr/bin/env ruby
  2. # == Synopsis
  3. #
  4. # This tool can exercize a puppetmaster by simulating an arbitraty number of concurrent clients
  5. # in a lightweight way.
  6. #
  7. # = Prerequisites
  8. #
  9. # This tool requires Event Machine and em-http-request, and an installation of Puppet.
  10. # Event Machine can be installed from gem.
  11. # em-http-request can be installed from gem.
  12. #
  13. # = Usage
  14. #
  15. #   puppet-load [-d|--debug] [--concurrency <num>] [--repeat <num>] [-V|--version] [-v|--verbose]
  16. #               [--node <host.domain.com>] [--facts <factfile>] [--cert <certfile>] [--key <keyfile>]
  17. #               [--factsdir <factsdir>] [--server <server.domain.com>]
  18. #
  19. # = Description
  20. #
  21. # This is a simple script meant for doing performance tests of puppet masters. It does this
  22. # by simulating concurrent connections to a puppet master and asking for catalog compilation.
  23. #
  24. # = Options
  25. #
  26. # Unlike other puppet executables, puppet-load doesn't parse puppet.conf nor use puppet options
  27. #
  28. # debug::
  29. #   Enable full debugging.
  30. #
  31. # concurreny::
  32. #   Number of simulated concurrent clients.
  33. #
  34. # server::
  35. #   Set the puppet master hostname or IP address..
  36. #
  37. # node::
  38. [root@master ext]# ./puppet-load.rb -d --timeout=10 --node client.abc.com --server master.abc.com --factsdir=/var/lib/puppet/yaml/facts --concurrency 1 --repeat 1 --cert /var/lib/puppet/ssl/certs/master.abc.com.pem --key /var/lib/puppet/ssl/private_keys/master.abc.com.pem
  39. debug: reading facts from: /var/lib/puppet/yaml/facts/client.abc.com.yaml
  40. debug: Client 0 finished with an error: Errno::ETIMEDOUT
  41. 1 requests finished in 5.322823 s
  42. 1 requests failed
  43. Availability: 0.00 %

  44. Time (s):
  45.         min: 5.322776 s
  46.         max: 5.322776 s
  47.         average: 5.322776 s
  48.         median: 5.322776 s

  49. Concurrency: 1.00
  50. Transaction Rate (tps): 0.19 t/s

  51. Received bytes: 0.00 B
  52. Throughput: 0.00000 MiB/s
  53. [root@master ext]# cat puppet-load.rb
  54. #!/usr/bin/env ruby
  55. # == Synopsis
  56. #
  57. # This tool can exercize a puppetmaster by simulating an arbitraty number of concurrent clients
  58. # in a lightweight way.
  59. #
  60. # = Prerequisites
  61. #
  62. # This tool requires Event Machine and em-http-request, and an installation of Puppet.
  63. # Event Machine can be installed from gem.
  64. # em-http-request can be installed from gem.
  65. #
  66. # = Usage
  67. #
  68. #   puppet-load [-d|--debug] [--concurrency <num>] [--repeat <num>] [-V|--version] [-v|--verbose]
  69. #               [--node <host.domain.com>] [--facts <factfile>] [--cert <certfile>] [--key <keyfile>]
  70. #               [--factsdir <factsdir>] [--server <server.domain.com>]
  71. #
  72. # = Description
  73. #
  74. # This is a simple script meant for doing performance tests of puppet masters. It does this
  75. # by simulating concurrent connections to a puppet master and asking for catalog compilation.
  76. #
  77. # = Options
  78. #
  79. # Unlike other puppet executables, puppet-load doesn't parse puppet.conf nor use puppet options
  80. #
  81. # debug::
  82. #   Enable full debugging.
  83. #
  84. # concurreny::
  85. #   Number of simulated concurrent clients.
  86. #
  87. # server::
  88. #   Set the puppet master hostname or IP address..
  89. #
  90. # node::
  91. #   Set the fully-qualified domain name of the client. This option can be given multiple
  92. #   times. In this case puppet-load will ask for catalog compilation of all the given nodes
  93. #   on a round robin way.
  94. #
  95. # help::
  96. #   Print this help message
  97. #
  98. # facts::
  99. #   This can be used to provide facts for the compilation, directly from a YAML
  100. #   file as found in the clientyaml directory. If none are provided, puppet-load
  101. #   will look by itself using Puppet facts indirector.
  102. #
  103. # factsdir::
  104. #   Specify a directory where the yaml facts files can be found. If provided puppet-load
  105. #   will look up facts in this directory. If not found it will resort to using Puppet Facts
  106. #   indirector.
  107. #
  108. # cert::
  109. #   This option is mandatory. It should be set to the cert PEM file that will be used
  110. #   to quthenticate the client connections.
  111. #
  112. # key::
  113. #   This option is mandatory. It should be set to the private key PEM file that will be used
  114. #   to quthenticate the client connections.
  115. #
  116. # timeout::
  117. #   The number of seconds after which a simulated client is declared in error if it didn't get
  118. #   a catalog. The default is 180s.
  119. #
  120. # repeat::
  121. #  How many times to perform the test. This means puppet-load will ask for
  122. #  concurrency * repeat catalogs.
  123. #
  124. # verbose::
  125. #   Turn on verbose reporting.
  126. #
  127. # version::
  128. #   Print the puppet version number and exit.
  129. #
  130. # = Example usage
  131. #
  132. # SINGLE NODE:
  133. #   1) On the master host, generate a new certificate and private key for our test host:
  134. #   puppet ca --generate puppet-load.domain.com
  135. #
  136. #   2) Copy the cert and key to the puppet-load host (which can be the same as the master one)
  137. #
  138. #   3) On the master host edit or create the auth.conf so that the catalog ACL match:
  139. #      path ~ ^/catalog/([^/]+)$
  140. #      method find
  141. #      allow $1
  142. #      allow puppet-load.domain.com
  143. #
  144. #   4) launch the master(s)
  145. #
  146. #   5) Prepare or get a fact file. One way to get one is to look on the master in $vardir/yaml/ for the host
  147. #   you want to simulate.
  148. #
  149. #   5) launch puppet-load
  150. #   puppet-load -debug --node server.domain.com --server master.domain.com --facts server.domain.com.yaml --concurrency 2 --repeat 20
  151. #
  152. # MULTIPLE NODES:
  153. #   1) On the master host, generate a new certificate and private key for our test host:
  154. #   puppet ca --generate puppet-load.domain.com
  155. #
  156. #   2) Copy the cert and key to the puppet-load host (which can be the same as the master one)
  157. #
  158. #   3) On the master host edit or create the auth.conf so that the catalog ACL match:
  159. #      path ~ ^/catalog/([^/]+)$
  160. #      method find
  161. #      allow $1
  162. #      allow puppet-load.domain.com
  163. #
  164. #   4) launch the master(s)
  165. #
  166. #   5) Prepare or get a fact file. One way to get one is to look on the master in $vardir/yaml/ for the host
  167. #   you want to simulate.
  168. #
  169. #   5) launch puppet-load
  170. #   puppet-load -debug --node server1.domain.com --node server2.domain.com --node server3.domain.com \
  171. #                      --server master.domain.com --factsdir /var/lib/puppet/yaml/facts --concurrency 2 --repeat 20
  172. #
  173. #   puppet-load will load facts file in the --factsdir directory based on the node name.
  174. #
  175. # = TODO
  176. #   * More output stats for error connections (ie report errors, HTTP code...)
  177. #
  178. #

  179. # Do an initial trap, so that cancels don't get a stack trace.
  180. trap(:INT) do
  181.   $stderr.puts "Cancelling startup"
  182.   exit(1)
  183. end

  184. require 'rubygems'
  185. require 'eventmachine'
  186. require 'em-http'
  187. require 'getoptlong'
  188. require 'puppet'

  189. $cmdargs = [
  190.   [ "--concurrency",  "-c", GetoptLong::REQUIRED_ARGUMENT       ],
  191.   [ "--node",     "-n", GetoptLong::REQUIRED_ARGUMENT ],
  192.   [ "--facts",          GetoptLong::REQUIRED_ARGUMENT ],
  193.   [ "--factsdir",       GetoptLong::REQUIRED_ARGUMENT ],
  194.   [ "--repeat",   "-r", GetoptLong::REQUIRED_ARGUMENT ],
  195.   [ "--cert",     "-C", GetoptLong::REQUIRED_ARGUMENT ],
  196.   [ "--key",      "-k", GetoptLong::REQUIRED_ARGUMENT ],
  197.   [ "--timeout",  "-t", GetoptLong::REQUIRED_ARGUMENT ],
  198.   [ "--server",   "-s", GetoptLong::REQUIRED_ARGUMENT ],
  199.   [ "--debug",    "-d", GetoptLong::NO_ARGUMENT       ],
  200.   [ "--help",     "-h", GetoptLong::NO_ARGUMENT       ],
  201.   [ "--verbose",  "-v", GetoptLong::NO_ARGUMENT       ],
  202.   [ "--version",  "-V", GetoptLong::NO_ARGUMENT       ],
  203. ]

  204. Puppet::Util::Log.newdestination(:console)

  205. times = {}

  206. def read_facts(file)
  207.   Puppet.debug("reading facts from: #{file}")
  208.   fact = YAML.load(File.read(file))
  209. end


  210. result = GetoptLong.new(*$cmdargs)

  211. $args = {}
  212. $options = {:repeat => 1, :concurrency => 1, :pause => false, :cert => nil, :key => nil, :timeout => 180, :masterport => 8140, :node => [], :factsdir => nil}

  213. begin
  214.   result.each { |opt,arg|
  215.     case opt
  216.     when "--concurrency"
  217.       begin
  218.         $options[:concurrency] = Integer(arg)
  219.       rescue => detail
  220.         $stderr.puts "The argument to 'fork' must be an integer"
  221.         exit(14)
  222.       end
  223.     when "--node"
  224.       $options[:node] << arg
  225.     when "--factsdir"
  226.       $options[:factsdir] = arg
  227.     when "--server"
  228.       $options[:server] = arg
  229.     when "--masterport"
  230.       $options[:masterport] = arg
  231.     when "--facts"
  232.       $options[:facts] = arg
  233.     when "--repeat"
  234.       $options[:repeat] = Integer(arg)
  235.     when "--help"
  236.       if Puppet.features.usage?
  237.         RDoc::usage && exit
  238.       else
  239.         puts "No help available unless you have RDoc::usage installed"
  240.         exit
  241.       end
  242.     when "--version"
  243.       puts "%s" % Puppet.version
  244.       exit
  245.     when "--verbose"
  246.       Puppet::Util::Log.level = :info
  247.       Puppet::Util::Log.newdestination(:console)
  248.     when "--debug"
  249.       Puppet::Util::Log.level = :debug
  250.       Puppet::Util::Log.newdestination(:console)
  251.     when "--cert"
  252.       $options[:cert] = arg
  253.     when "--key"
  254.       $options[:key] = arg
  255.     end
  256.   }
  257. rescue GetoptLong::InvalidOption => detail
  258.   $stderr.puts detail
  259.   $stderr.puts "Try '#{$0} --help'"
  260.   exit(1)
  261. end

  262. unless $options[:cert] and $options[:key]
  263.   raise "--cert and --key are mandatory to authenticate the client"
  264. end

  265. parameters = []

  266. unless $options[:node].size > 0
  267.   raise "--node is a mandatory argument. It tells to the master what node to compile"
  268. end

  269. $options[:node].each do |node|
  270.   factfile = $options[:factsdir] ? File.join($options[:factsdir], node + ".yaml") : $options[:facts]
  271.   unless fact = read_facts(factfile) or fact = Puppet::Node::Facts.find(node)
  272.     raise "Could not find facts for %s" % node
  273.   end
  274.   fact.values["fqdn"] = node
  275.   fact.values["hostname"] = node.sub(/\..+/, '')
  276.   fact.values["domain"] = node.sub(/^[^.]+\./, '')

  277.   parameters << {:facts_format => "b64_zlib_yaml", :facts => CGI.escape(fact.render(:b64_zlib_yaml))}
  278. end


  279. class RequestPool
  280.   include EventMachine::Deferrable

  281.   attr_reader :requests, :responses, :times, :sizes
  282.   attr_reader :repeat, :concurrency, :max_request

  283.   def initialize(concurrency, repeat, parameters)
  284.     @parameters = parameters
  285.     @current_request = 0
  286.     @max_request = repeat * concurrency
  287.     @repeat = repeat
  288.     @concurrency = concurrency
  289.     @requests = []
  290.     @responses = {:succeeded => [], :failed => []}
  291.     @times = {}
  292.     @sizes = {}

  293.     # initial spawn
  294.     (1..concurrency).each do |i|
  295.       spawn
  296.     end

  297.   end

  298.   def spawn_request(index)
  299.     @times[index] = Time.now
  300.     @sizes[index] = 0
  301.     nodeidx = index % $options[:node].size
  302.     node = $options[:node][nodeidx]
  303.     EventMachine::HttpRequest.new("https://#{$options[:server]}:#{$options[:masterport]}/production/catalog/#{node}").get(
  304.     :port => $options[:masterport],
  305.     :query => @parameters[nodeidx],
  306.     :timeout => $options[:timeout],
  307.     :head => { "Accept" => "pson, yaml, b64_zlib_yaml, marshal, dot, raw", "Accept-Encoding" => "gzip, deflate" },
  308.     :ssl => { :private_key_file => $options[:key],
  309.               :cert_chain_file => $options[:cert],
  310.               :verify_peer => false } ) do
  311.         @times[index] = Time.now
  312.         @sizes[index] = 0
  313.         Puppet.debug("starting client #{index} for #{node}")
  314.     end
  315.   end

  316.   def add(index, conn)
  317.     @requests.push(conn)

  318.     conn.stream { |data|
  319.       @sizes[index] += data.length
  320.     }

  321.     conn.callback {
  322.       @times[index] = Time.now - @times[index]
  323.       code = conn.response_header.status
  324.       if code >= 200 && code < 300
  325.         Puppet.debug("Client #{index} finished successfully")
  326.         @responses[:succeeded].push(conn)
  327.       else
  328.         Puppet.debug("Client #{index} finished with HTTP code #{code}")
  329.         @responses[:failed].push(conn)
  330.       end
  331.       check_progress
  332.     }

  333.     conn.errback {
  334.       Puppet.debug("Client #{index} finished with an error: #{conn.error}")
  335.       @times[index] = Time.now - @times[index]
  336.       @responses[:failed].push(conn)
  337.       check_progress
  338.     }
  339.   end

  340.   def all_responses
  341.     @responses[:succeeded] + @responses[:failed]
  342.   end

  343.   protected

  344.   def check_progress
  345.     spawn unless all_spawned?
  346.     succeed if all_finished?
  347.   end

  348.   def all_spawned?
  349.     @requests.size >= max_request
  350.   end

  351.   def all_finished?
  352.     @responses[:failed].size + @responses[:succeeded].size >= max_request
  353.   end

  354.   def spawn
  355.     add(@current_request, spawn_request(@current_request))
  356.     @current_request += 1
  357.   end
  358. end


  359. def mean(array)
  360.   array.inject(0) { |sum, x| sum += x } / array.size.to_f
  361. end

  362. def median(array)
  363.   array = array.sort
  364.   m_pos = array.size / 2
  365.   return array.size % 2 == 1 ? array[m_pos] : mean(array[m_pos-1..m_pos])
  366. end

  367. def format_bytes(bytes)
  368.   if bytes < 1024
  369.     "%.2f B" % bytes
  370.   elsif bytes < 1024 * 1024
  371.     "%.2f KiB" % (bytes/1024.0)
  372.   else
  373.     "%.2f MiB" % (bytes/(1024.0*1024.0))
  374.   end
  375. end

  376. EM::run {

  377.   start = Time.now
  378.   multi = RequestPool.new($options[:concurrency], $options[:repeat], parameters)

  379.   multi.callback do
  380.     duration = Time.now - start
  381.     puts "#{multi.max_request} requests finished in #{duration} s"
  382.     puts "#{multi.responses[:failed].size} requests failed"
  383.     puts "Availability: %3.2f %%" % (100.0*multi.responses[:succeeded].size/(multi.responses[:succeeded].size+multi.responses[:failed].size))

  384.     minmax = multi.times.values.minmax
  385.     all_time = multi.times.values.reduce(:+)

  386.     puts "\nTime (s):"
  387.     puts "\tmin: #{minmax[0]} s"
  388.     puts "\tmax: #{minmax[1]} s"
  389.     puts "\taverage: #{mean(multi.times.values)} s"
  390.     puts "\tmedian: #{median(multi.times.values)} s"

  391.     puts "\nConcurrency: %.2f" % (all_time/duration)
  392.     puts "Transaction Rate (tps): %.2f t/s" % (multi.max_request / duration)

  393.     transferred = multi.sizes.values.reduce(:+)

  394.     puts "\nReceived bytes: #{format_bytes(transferred)}"
  395.     puts "Throughput: %.5f MiB/s" % (transferred/duration/(1024.0*1024.0))

  396.     # this is the end
  397.     EventMachine.stop
  398.   end
  399. }
复制代码
我执行报这个错
  1. [root@master ext]# ./puppet-load.rb -d  --node client.abc.com --server master.abc.com --factsdir=/var/lib/puppet/yaml/facts --concurrency 1 --repeat 1 --cert /var/lib/puppet/ssl/certs/master.abc.com.pem --key /var/lib/puppet/ssl/private_keys/master.abc.com.pem              debug: reading facts from: /var/lib/puppet/yaml/facts/client.abc.com.yaml
  2. debug: Client 0 finished with an error: Errno::ETIMEDOUT
  3. 1 requests finished in 5.00397 s
  4. 1 requests failed
  5. Availability: 0.00 %

  6. Time (s):
  7.         min: 5.003886 s
  8.         max: 5.003886 s
  9.         average: 5.003886 s
  10.         median: 5.003886 s

  11. Concurrency: 1.00
  12. Transaction Rate (tps): 0.20 t/s

  13. Received bytes: 0.00 B
  14. Throughput: 0.00000 MiB/s
复制代码
这个 Errno::ETIMEDOUT 报错 是什么意思?要怎么解决。。
您需要登录后才可以回帖 登录 | 注册

本版积分规则 发表回复

  

北京盛拓优讯信息技术有限公司. 版权所有 京ICP备16024965号-6 北京市公安局海淀分局网监中心备案编号:11010802020122 niuxiaotong@pcpop.com 17352615567
未成年举报专区
中国互联网协会会员  联系我们:huangweiwei@itpub.net
感谢所有关心和支持过ChinaUnix的朋友们 转载本站内容请注明原作者名及出处

清除 Cookies - ChinaUnix - Archiver - WAP - TOP