提供了编程的基础技术教程

zl程序教程

spark例子

  • spark pipeline 例子

    spark pipeline 例子

      """ Pipeline Example. """ # $example on$ from pyspark.ml import Pipeline from pyspark.ml.classification import LogisticRegression from pyspark.ml.feature import HashingTF, Tokenizer # $examp

    日期 2023-06-12 10:48:40     
  • spark streaming 入门例子

    spark streaming 入门例子

    spark streaming 入门例子： spark shell import org.apache.spark._ import org.apache.spark.streaming._ sc.getConf.setMaster("local").setAppName("RDDTest"); val ssc = new StreamingContext(sc, Seconds(2));

    日期 2023-06-12 10:48:40     
  • [Spark][Python]DataFrame select 操作例子

    [Spark][Python]DataFrame select 操作例子

    [Spark][Python]DataFrame中取出有限个记录的例子  的 继续 In [4]: peopleDF.select("age")Out[4]: DataFrame[age: bigint] In [5]: myDF=people.select("age")----------------------------------------------------------

    日期 2023-06-12 10:48:40     
  • [Spark][Python][DataFrame][SQL]Spark对DataFrame直接执行SQL处理的例子

    [Spark][Python][DataFrame][SQL]Spark对DataFrame直接执行SQL处理的例子

    [Spark][Python][DataFrame][SQL]Spark对DataFrame直接执行SQL处理的例子 $cat people.json {"name":"Alice","pcode":"94304"}{"name":"Brayden","age":30,"pcode":"94304"}{"name":"Carla","age":19,"pcoe":"10036"}{"name"

    日期 2023-06-12 10:48:40     
  • [python][spark]wholeTextFiles 读入多个文件的例子

    [python][spark]wholeTextFiles 读入多个文件的例子

    $pwd  /home/training/mydir $cat file1.json {"firstName":"Fred", "lastName":"Flintstone", "userid":"123"} $cat file2.json {"firstName":"Barney", "lastName":"Rubble", "userid":"123"} [training@loc

    日期 2023-06-12 10:48:40     
  • [Spark][Python]RDD flatMap 操作例子

    [Spark][Python]RDD flatMap 操作例子

    RDD flatMap 操作例子： flatMap，对原RDD的每个元素(行)执行函数操作，然后把每行都“拍扁” [training@localhost ~]$ hdfs dfs -put cats.txt[training@localhost ~]$ hdfs dfa -cat cats.txtError: Could not find or load main class dfa[train

    日期 2023-06-12 10:48:40