Remember Me
Or use your Academic/Social account:


Or use your Academic/Social account:


You have just completed your registration at OpenAire.

Before you can login to the site, you will need to activate your account. An e-mail will be sent to you with the proper instructions.


Please note that this site is currently undergoing Beta testing.
Any new content you create is not guaranteed to be present to the final version of the site upon release.

Thank you for your patience,
OpenAire Dev Team.

Close This Message


Verify Password:
Verify E-mail:
*All Fields Are Required.
Please Verify You Are Human:
fbtwitterlinkedinvimeoflicker grey 14rssslideshare1
Chen, Huankai; Wang, Frank Z. (2015)
Publisher: IEEE
Languages: English
Types: Unknown
Subjects: QA
In heterogeneous cloud, the provision of quality of\ud service (QoS) guarantees for on-line parallel analysis jobs is much\ud more challenging than off-line ones, mainly due to the many\ud involved parameters, unstable resource performance, various job\ud pattern and dynamic query workload. In this paper we propose\ud an entropy-based scheduling strategy for running the on-line\ud parallel analysis as a service more reliable and efficient, and\ud implement the proposed idea in Spark.\ud Entropy, as a measure of the degree of disorder in a system,\ud is an indicator of a system’s tendency to progress out of order\ud and into a chaotic condition, and it can thus serve to measure a\ud cloud resource’s reliability for jobs scheduling. The key idea of\ud our Entropy Scheduler is to construct the new resource entropy\ud metric and schedule tasks according to the resources ranking with\ud the help of the new metric so as to provide QoS guarantees for\ud on-line Spark analysis jobs. Experiments demonstrate that our\ud approach significantly reduces the average query response time\ud by 15% - 20% and standard deviation by 30% - 45% compare\ud with the native Fair Scheduler in Spark.
  • The results below are discovered through our pilot algorithms. Let us know how we are doing!

    • [1] Amos, Brandon, and David Tompkins. ”Performance Study of Spindle, A Web Analytics Query Engine Implemented in Spark.” Cloud Computing Technology and Science (CloudCom), 2014 IEEE 6th International Conference on. IEEE, 2014.
    • [2] Ousterhout, Kay, et al. ”Sparrow: distributed, low latency scheduling.” Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles. ACM, 2013.
    • ”Entropy-based scheduling of resource-constrained construction projects.” Automation in Construction 18.7 (2009): 919-928.
    • Hermenier, Fabien, et al. ”Entropy: a consolidation manager for clusters.” Proceedings of the 2009 ACM SIGPLAN/SIGOPS international conference on Virtual execution environments. ACM, 2009.
    • Chua, Leon O. ”Local activity is the origin of complexity.” International journal of bifurcation and chaos 15.11 (2005): 3435-3456.
    • Zaharia, Matei, et al. ”Improving MapReduce Performance in Heterogeneous Environments.” OSDI. Vol. 8. No. 4. 2008.
    • Zaharia, Matei, et al. ”Spark: cluster computing with working sets.” Proceedings of the 2nd USENIX conference on Hot topics in cloud computing. 2010.
    • Schwarzkopf, Malte, et al. ”Omega: flexible, scalable schedulers for large compute clusters.” Proceedings of the 8th ACM European Conference on Computer Systems. ACM, 2013.
    • Hindman, Benjamin, et al. ”Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center.” NSDI. Vol. 11. 2011.
    • Vavilapalli, Vinod Kumar, et al. ”Apache hadoop yarn: Yet another resource negotiator.” Proceedings of the 4th annual Symposium on Cloud Computing. ACM, 2013.
    • Matthews, Robert AJ. ”The science of Murphy's Law.” SCIENTIFIC AMERICAN-AMERICAN EDITION- 276 (1997): 88-91.
    • Plestys, R., et al. ”The measurement of grid QoS parameters.” Information Technology Interfaces, 2007. ITI 2007. 29th International Conference on. IEEE, 2007.
    • Gan, H-S., and A. Wirth. ”Comparing deterministic, robust and online scheduling using entropy.” International journal of production research 43.10 (2005): 2113-2134.
    • Botn-Fernndez, Mara, Francisco Prieto Castrillo, and Miguel VegaRodrguez. ”Nature-inspired algorithms applied to an efficient and self-adaptive resources selection model for grid Practice of Natural Computing (2012): 84-96.
    • Wolfram, Stephen. ”Universality and complexity in cellular automata.” Physica D: Nonlinear Phenomena 10.1 (1984): 1-35.
    • LEON, O. ”Local activity is the origin of complexity.” International journal of bifurcation and chaos 15.11 (2005): 3435-3456.
    • Matthews, Robert AJ. ”The science of Murphy's Law.” SCIENTIFIC AMERICAN-AMERICAN EDITION- 276 (1997): 88-91.
    • Lambert, F.L.: The Second http://www.secondlaw.com, 2005.
  • No related research data.
  • No similar publications.

Share - Bookmark

Cite this article