您现在的位置是：首页 > Java

当前栏目

java scanner getline,Java PublicScanner類代碼示例

2023-03-02 11:48:29 时间

本文整理匯總了Java中org.eclipse.jdt.internal.core.util.PublicScanner類的典型用法代碼示例。如果您正苦於以下問題：Java PublicScanner類的具體用法？Java PublicScanner怎麽用？Java PublicScanner使用的例子？那麽恭喜您, 這裏精選的類代碼示例或許可以為您提供幫助。

PublicScanner類屬於org.eclipse.jdt.internal.core.util包，在下文中一共展示了PublicScanner類的18個代碼示例，這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚，您的評價將有助於我們的係統推薦出更棒的Java代碼示例。

示例1: getIdentifierType

點讚 3

import org.eclipse.jdt.internal.core.util.PublicScanner; //導入依賴的package包/類

public String getIdentifierType(final PublicScanner scanner) \{

final int startPos = scanner.getCurrentTokenStartPosition();

final int endPos = scanner.getCurrentTokenEndPosition();

final Range tokenRange = Range

.closedOpen(startPos, endPos);

final String tokenName = scanner.getCurrentTokenString();

// TODO: Find the tightest of all

if (isInSet(tokenName, tokenRange, variables)) \{

return IDENTIFIER\_PREFIX + "\_VAR";

\} else if (isInSet(tokenName, tokenRange, methods)) \{

return IDENTIFIER\_PREFIX + "\_METHOD";

\} else if (isInSet(tokenName, tokenRange, types)) \{

return IDENTIFIER\_PREFIX + "\_TYPE";

return IDENTIFIER\_PREFIX + "\_UNK";

開發者ID:mast-group，項目名稱:tassal，代碼行數:19，

示例2: getTokenListFromCode

點讚 3

import org.eclipse.jdt.internal.core.util.PublicScanner; //導入依賴的package包/類

@Override

public List getTokenListFromCode(final char\[\] code) \{

final List tokens = Lists.newArrayList();

tokens.add(new FullToken(SENTENCE\_START, SENTENCE\_START));

final PublicScanner scanner = prepareScanner(code);

do \{

try \{

final int token = scanner.getNextToken();

if (token == ITerminalSymbols.TokenNameEOF) \{

break;

for (final String cToken : getConvertedToken(scanner, token)) \{

tokens.add(new FullToken(cToken, ""));

\} catch (final InvalidInputException e) \{

LOGGER.warning(ExceptionUtils.getFullStackTrace(e));

\} while (!scanner.atEnd());

tokens.add(new FullToken(SENTENCE\_END, SENTENCE\_END));

return tokens;

開發者ID:mast-group，項目名稱:tassal，代碼行數:22，

示例3: tokenListFromCode

點讚 3

import org.eclipse.jdt.internal.core.util.PublicScanner; //導入依賴的package包/類

@Override

public List tokenListFromCode(final char\[\] code) \{

final List tokens = Lists.newArrayList();

tokens.add(SENTENCE\_START);

final PublicScanner scanner = prepareScanner(code);

do \{

try \{

final int token = scanner.getNextToken();

if (token == ITerminalSymbols.TokenNameEOF) \{

break;

tokens.addAll(getConvertedToken(scanner, token));

\} catch (final InvalidInputException e) \{

LOGGER.warning(ExceptionUtils.getFullStackTrace(e));

\} while (!scanner.atEnd());

tokens.add(SENTENCE\_END);

return tokens;

開發者ID:mast-group，項目名稱:tassal，代碼行數:20，

示例4: tokenListWithPos

點讚 3

import org.eclipse.jdt.internal.core.util.PublicScanner; //導入依賴的package包/類

@Override

public SortedMap tokenListWithPos(final char\[\] code) \{

final PublicScanner scanner = prepareScanner();

final SortedMap tokens = Maps.newTreeMap();

tokens.put(-1, SENTENCE\_START);

tokens.put(Integer.MAX\_VALUE, SENTENCE\_END);

scanner.setSource(code);

while (!scanner.atEnd()) \{

do \{

try \{

final int token = scanner.getNextToken();

if (token == ITerminalSymbols.TokenNameEOF) \{

break;

final String nxtToken = transformToken(token,

scanner.getCurrentTokenString());

final int position = scanner.getCurrentTokenStartPosition();

tokens.put(position, stripTokenIfNeeded(nxtToken));

\} catch (final InvalidInputException e) \{

LOGGER.warning(ExceptionUtils.getFullStackTrace(e));

\} while (!scanner.atEnd());

return tokens;

開發者ID:mast-group，項目名稱:tassal，代碼行數:27，

示例5: getTokenType

點讚 2

import org.eclipse.jdt.internal.core.util.PublicScanner; //導入依賴的package包/類

private final String getTokenType(final int tokenType,

final PublicScanner scanner, final IdentifierTypeRetriever retriever) \{

if (tokenType == ITerminalSymbols.TokenNameIdentifier) \{

return retriever.getIdentifierType(scanner);

\} else if (JavaTokenTypeTokenizer.isLiteralToken(tokenType)) \{

return LITERAL;

\} else \{

return scanner.getCurrentTokenString();

開發者ID:mast-group，項目名稱:tassal，代碼行數:11，

示例6: fullTokenListWithPos

點讚 2

import org.eclipse.jdt.internal.core.util.PublicScanner; //導入依賴的package包/類

@Override

public SortedMap fullTokenListWithPos(final char\[\] code) \{

final SortedMap tokens = Maps.newTreeMap();

tokens.put(-1, new FullToken(SENTENCE\_START, SENTENCE\_START));

tokens.put(Integer.MAX\_VALUE, new FullToken(SENTENCE\_END, SENTENCE\_END));

final PublicScanner scanner = createScanner();

scanner.setSource(code);

while (!scanner.atEnd()) \{

do \{

try \{

final int token = scanner.getNextToken();

final int position = scanner.getCurrentTokenStartPosition();

if (token == ITerminalSymbols.TokenNameEOF) \{

break;

\} else if (token == ITerminalSymbols.TokenNameIdentifier) \{

tokens.put(position,

new FullToken(IDENTIFIER\_TOKEN, ""));

\} else if (token == ITerminalSymbols.TokenNameCOMMENT\_BLOCK) \{

tokens.put(position, new FullToken(COMMENT\_BLOCK, ""));

\} else if (token == ITerminalSymbols.TokenNameCOMMENT\_JAVADOC) \{

tokens.put(position, new FullToken(COMMENT\_JAVADOC, ""));

\} else if (token == ITerminalSymbols.TokenNameCOMMENT\_LINE) \{

tokens.put(position, new FullToken(COMMENT\_LINE, ""));

\} else if (isLiteralToken(token)) \{

tokens.put(position, new FullToken(LITERAL\_TOKEN, ""));

\} else \{

tokens.put(position,

new FullToken(scanner.getCurrentTokenString(),

""));

\} catch (final InvalidInputException e) \{

LOGGER.warning(ExceptionUtils.getFullStackTrace(e));

\} while (!scanner.atEnd());

return tokens;

開發者ID:mast-group，項目名稱:tassal，代碼行數:41，

示例7: getTokenListFromCode

點讚 2

import org.eclipse.jdt.internal.core.util.PublicScanner; //導入依賴的package包/類

@Override

public List getTokenListFromCode(final char\[\] code) \{

final List tokens = Lists.newArrayList();

tokens.add(new FullToken(SENTENCE\_START, SENTENCE\_START));

final PublicScanner scanner = createScanner();

scanner.setSource(code);

do \{

try \{

final int token = scanner.getNextToken();

if (token == ITerminalSymbols.TokenNameEOF) \{

break;

\} else if (token == ITerminalSymbols.TokenNameIdentifier) \{

tokens.add(new FullToken(IDENTIFIER\_TOKEN, ""));

\} else if (isLiteralToken(token)) \{

tokens.add(new FullToken(LITERAL\_TOKEN, ""));

\} else if (token == ITerminalSymbols.TokenNameCOMMENT\_BLOCK) \{

tokens.add(new FullToken(COMMENT\_BLOCK, ""));

\} else if (token == ITerminalSymbols.TokenNameCOMMENT\_JAVADOC) \{

tokens.add(new FullToken(COMMENT\_JAVADOC, ""));

\} else if (token == ITerminalSymbols.TokenNameCOMMENT\_LINE) \{

tokens.add(new FullToken(COMMENT\_LINE, ""));

\} else \{

tokens.add(new FullToken(scanner.getCurrentTokenString(),

""));

\} catch (final InvalidInputException e) \{

LOGGER.warning(ExceptionUtils.getFullStackTrace(e));

\} while (!scanner.atEnd());

tokens.add(new FullToken(SENTENCE\_END, SENTENCE\_END));

return tokens;

開發者ID:mast-group，項目名稱:tassal，代碼行數:34，

示例8: tokenListFromCode

點讚 2

import org.eclipse.jdt.internal.core.util.PublicScanner; //導入依賴的package包/類

@Override

public List tokenListFromCode(final char\[\] code) \{

final List tokens = Lists.newArrayList();

tokens.add(SENTENCE\_START);

final PublicScanner scanner = createScanner();

scanner.setSource(code);

do \{

try \{

final int token = scanner.getNextToken();

if (token == ITerminalSymbols.TokenNameEOF) \{

break;

\} else if (token == ITerminalSymbols.TokenNameIdentifier) \{

tokens.add(IDENTIFIER\_TOKEN);

\} else if (token == ITerminalSymbols.TokenNameCOMMENT\_BLOCK) \{

tokens.add(COMMENT\_BLOCK);

\} else if (token == ITerminalSymbols.TokenNameCOMMENT\_LINE) \{

tokens.add(COMMENT\_LINE);

\} else if (token == ITerminalSymbols.TokenNameCOMMENT\_JAVADOC) \{

tokens.add(COMMENT\_JAVADOC);

\} else if (isLiteralToken(token)) \{

tokens.add(LITERAL\_TOKEN);

\} else \{

tokens.add(scanner.getCurrentTokenString());

\} catch (final InvalidInputException e) \{

LOGGER.warning(ExceptionUtils.getFullStackTrace(e));

\} while (!scanner.atEnd());

tokens.add(SENTENCE\_END);

return tokens;

開發者ID:mast-group，項目名稱:tassal，代碼行數:33，

示例9: tokenListWithPos

點讚 2

import org.eclipse.jdt.internal.core.util.PublicScanner; //導入依賴的package包/類

@Override

public SortedMap tokenListWithPos(final char\[\] code) \{

final SortedMap tokens = Maps.newTreeMap();

tokens.put(-1, SENTENCE\_START);

tokens.put(Integer.MAX\_VALUE, SENTENCE\_END);

final PublicScanner scanner = createScanner();

scanner.setSource(code);

while (!scanner.atEnd()) \{

do \{

try \{

final int token = scanner.getNextToken();

final int position = scanner.getCurrentTokenStartPosition();

if (token == ITerminalSymbols.TokenNameEOF) \{

break;

\} else if (token == ITerminalSymbols.TokenNameIdentifier) \{

tokens.put(position, IDENTIFIER\_TOKEN);

\} else if (isLiteralToken(token)) \{

tokens.put(position, LITERAL\_TOKEN);

\} else if (token == ITerminalSymbols.TokenNameCOMMENT\_BLOCK) \{

tokens.put(position, COMMENT\_BLOCK);

\} else if (token == ITerminalSymbols.TokenNameCOMMENT\_JAVADOC) \{

tokens.put(position, COMMENT\_JAVADOC);

\} else if (token == ITerminalSymbols.TokenNameCOMMENT\_LINE) \{

tokens.put(position, COMMENT\_LINE);

\} else \{

tokens.put(position, scanner.getCurrentTokenString());

\} catch (final InvalidInputException e) \{

LOGGER.warning(ExceptionUtils.getFullStackTrace(e));

\} while (!scanner.atEnd());

return tokens;

開發者ID:mast-group，項目名稱:tassal，代碼行數:38，

示例10: getConvertedToken

點讚 2

import org.eclipse.jdt.internal.core.util.PublicScanner; //導入依賴的package包/類

/\*\*

\* @param tokens

\* @param scanner

\* @param token

\* @return

\* @throws InvalidInputException

\*/

private List getConvertedToken(final PublicScanner scanner,

final int token) throws InvalidInputException \{

final List tokens = Lists.newArrayList();

if (token == ITerminalSymbols.TokenNameEOF) \{

return Collections.emptyList();

final String tokenString = scanner.getCurrentTokenString();

if (token == ITerminalSymbols.TokenNameWHITESPACE) \{

tokens.add(whitespaceConverter.toWhiteSpaceSymbol(tokenString));

\} else if (token == ITerminalSymbols.TokenNameIdentifier) \{

tokens.add(toIdentifierSymbol(tokenString));

\} else if (JavaTokenTypeTokenizer.isLiteralToken(token)) \{

tokens.add(toLiteralSymbol(tokenString));

\} else if (token == ITerminalSymbols.TokenNameCOMMENT\_BLOCK) \{

tokens.add(JavaTokenTypeTokenizer.COMMENT\_BLOCK);

\} else if (token == ITerminalSymbols.TokenNameCOMMENT\_LINE) \{

tokens.add(JavaTokenTypeTokenizer.COMMENT\_LINE);

final int nextToken = scanner.getNextToken();

if (nextToken == ITerminalSymbols.TokenNameWHITESPACE) \{

tokens.add(whitespaceConverter.toWhiteSpaceSymbol("\\n"

\+ scanner.getCurrentTokenString()));

\} else \{

tokens.add(whitespaceConverter.toWhiteSpaceSymbol("\\n"));

tokens.addAll(getConvertedToken(scanner, nextToken));

\} else if (token == ITerminalSymbols.TokenNameCOMMENT\_JAVADOC) \{

tokens.add(JavaTokenTypeTokenizer.COMMENT\_JAVADOC);

\} else \{

tokens.add(tokenString);

return tokens;

開發者ID:mast-group，項目名稱:tassal，代碼行數:42，

示例11: getTokensWithWidthData

點讚 2

import org.eclipse.jdt.internal.core.util.PublicScanner; //導入依賴的package包/類

public List getTokensWithWidthData(

final char\[\] code) \{

final List tokens = Lists.newArrayList();

tokens.add(new WhitespaceAnnotatedToken(SENTENCE\_START,

SENTENCE\_START, 0, 0));

final PublicScanner scanner = prepareScanner(code);

do \{

try \{

final int token = scanner.getNextToken();

if (token == ITerminalSymbols.TokenNameEOF) \{

break;

for (final String cToken : getConvertedToken(scanner, token)) \{

final int currentPosition = scanner

.getCurrentTokenStartPosition();

final int currentLine = scanner

.getLineNumber(currentPosition);

final int lineStart = scanner.getLineStart(currentLine);

tokens.add(new WhitespaceAnnotatedToken(cToken, "",

currentPosition - lineStart, scanner

.getCurrentTokenString().length()));

\} catch (final InvalidInputException e) \{

LOGGER.warning(ExceptionUtils.getFullStackTrace(e));

\} while (!scanner.atEnd());

tokens.add(new WhitespaceAnnotatedToken(SENTENCE\_END, SENTENCE\_END,

0, 0));

return tokens;

開發者ID:mast-group，項目名稱:tassal，代碼行數:31，

示例12: prepareScanner

點讚 2

import org.eclipse.jdt.internal.core.util.PublicScanner; //導入依賴的package包/類

/\*\*

\* @param code

\* @return

\*/

private PublicScanner prepareScanner(final char\[\] code) \{

final PublicScanner scanner = new PublicScanner();

scanner.setSource(code);

scanner.tokenizeWhiteSpace = true;

scanner.recordLineSeparator = true;

scanner.tokenizeComments = true;

whitespaceConverter = new WhitespaceToTokenConverter();

return scanner;

開發者ID:mast-group，項目名稱:tassal，代碼行數:14，

示例13: tokenListWithPos

點讚 2

import org.eclipse.jdt.internal.core.util.PublicScanner; //導入依賴的package包/類

@Override

public SortedMap tokenListWithPos(final char\[\] code) \{

final SortedMap tokens = Maps.newTreeMap();

tokens.put(-1, SENTENCE\_START);

tokens.put(Integer.MAX\_VALUE, SENTENCE\_END);

final PublicScanner scanner = prepareScanner(code);

while (!scanner.atEnd()) \{

do \{

try \{

final int token = scanner.getNextToken();

final int position = scanner

.getCurrentTokenStartPosition();

if (token == ITerminalSymbols.TokenNameEOF) \{

break;

int i = 0;

final List cTokens = getConvertedToken(scanner,

token);

for (final String cToken : cTokens) \{

tokens.put(position + i, cToken);

i++;

\} catch (final InvalidInputException e) \{

LOGGER.warning(ExceptionUtils.getFullStackTrace(e));

\} while (!scanner.atEnd());

return tokens;

開發者ID:mast-group，項目名稱:tassal，代碼行數:31，

示例14: fullTokenListWithPos

點讚 2

import org.eclipse.jdt.internal.core.util.PublicScanner; //導入依賴的package包/類

@Override

public SortedMap fullTokenListWithPos(final char\[\] code) \{

// TODO Duplicate Code

final PublicScanner scanner = prepareScanner();

final SortedMap tokens = Maps.newTreeMap();

tokens.put(-1, new FullToken(SENTENCE\_START, SENTENCE\_START));

tokens.put(Integer.MAX\_VALUE, new FullToken(SENTENCE\_END, SENTENCE\_END));

scanner.setSource(code);

while (!scanner.atEnd()) \{

do \{

try \{

final int token = scanner.getNextToken();

if (token == ITerminalSymbols.TokenNameEOF) \{

break;

final String nxtToken = transformToken(token,

scanner.getCurrentTokenString());

final int position = scanner.getCurrentTokenStartPosition();

tokens.put(position,

new FullToken(nxtToken, Integer.toString(token)));

\} catch (final InvalidInputException e) \{

LOGGER.warning(ExceptionUtils.getFullStackTrace(e));

\} while (!scanner.atEnd());

return tokens;

開發者ID:mast-group，項目名稱:tassal，代碼行數:29，

示例15: createScanner

點讚 2

import org.eclipse.jdt.internal.core.util.PublicScanner; //導入依賴的package包/類

/\*\*

\* @return

\*/

private PublicScanner createScanner() \{

final PublicScanner scanner = new PublicScanner();

scanner.tokenizeComments = tokenizeComments;

return scanner;

開發者ID:mast-group，項目名稱:tassal，代碼行數:9，

示例16: tokenListWithPosAndWidth

點讚 2

import org.eclipse.jdt.internal.core.util.PublicScanner; //導入依賴的package包/類

public SortedMap tokenListWithPosAndWidth(

final char\[\] code) \{

final SortedMap tokens = Maps

.newTreeMap();

tokens.put(-1, new WhitespaceAnnotatedToken(SENTENCE\_START,

SENTENCE\_START, 0, 0));

tokens.put(Integer.MAX\_VALUE, new WhitespaceAnnotatedToken(

SENTENCE\_END, SENTENCE\_END, 0, 0));

final PublicScanner scanner = prepareScanner(code);

while (!scanner.atEnd()) \{

do \{

try \{

final int token = scanner.getNextToken();

final int currentPosition = scanner

.getCurrentTokenStartPosition();

final int currentLine = scanner

.getLineNumber(currentPosition);

final int lineStart = scanner.getLineStart(currentLine);

final int position = scanner

.getCurrentTokenStartPosition();

if (token == ITerminalSymbols.TokenNameEOF) \{

break;

int i = 0;

final List cTokens = getConvertedToken(scanner,

token);

for (final String cToken : cTokens) \{

tokens.put(position + i,

new WhitespaceAnnotatedToken(cToken, "",

currentPosition - lineStart,

scanner.getCurrentTokenString()

.length()));

i++;

\} catch (final InvalidInputException e) \{

LOGGER.warning(ExceptionUtils.getFullStackTrace(e));

\} while (!scanner.atEnd());

return tokens;

開發者ID:mast-group，項目名稱:tassal，代碼行數:43，

示例17: prepareScanner

點讚 2

import org.eclipse.jdt.internal.core.util.PublicScanner; //導入依賴的package包/類

/\*\*

\* Create the scanner.

\* @return

\*/

protected PublicScanner prepareScanner() \{

final PublicScanner scanner = new PublicScanner();

scanner.tokenizeComments = tokenizeComments;

return scanner;

開發者ID:mast-group，項目名稱:tassal，代碼行數:11，

示例18: createScanner

點讚 2

import org.eclipse.jdt.internal.core.util.PublicScanner; //導入依賴的package包/類

/\*\*

\* Create a scanner, indicating the level of detail requested for tokenizing. The scanner can then be

\* used to tokenize some source in a Java aware way.

\* Here is a typical scanning loop:

\* ``

\* IScanner scanner = ToolFactory.createScanner(false, false, false, false);

\* scanner.setSource("int i = 0;".toCharArray());

\* while (true) \{

\* int token = scanner.getNextToken();

\* if (token == ITerminalSymbols.TokenNameEOF) break;

\* System.out.println(token + " : " + new String(scanner.getCurrentTokenSource()));

\* \}

By default the compliance used to create the scanner is the workspace's compliance when running inside the IDE

\* or 1.4 if running from outside of a headless eclipse.

\* @param tokenizeComments if set to `false`, comments will be silently consumed

\* @param tokenizeWhiteSpace if set to `false`, white spaces will be silently consumed,

\* @param assertMode if set to `false`, occurrences of 'assert' will be reported as identifiers

\* (\{@link ITerminalSymbols\#TokenNameIdentifier\}), whereas if set to `true`, it

\* would report assert keywords (\{@link ITerminalSymbols\#TokenNameassert\}). Java 1.4 has introduced

\* a new 'assert' keyword.

\* @param recordLineSeparator if set to `true`, the scanner will record positions of encountered line

\* separator ends. In case of multi-character line separators, the last character position is considered. These positions

\* can then be extracted using \{@link IScanner\#getLineEnds()\}. Only non-unicode escape sequences are

\* considered as valid line separators.

\* @return a scanner

\* @see org.eclipse.jdt.core.compiler.IScanner

\* @see \#createScanner(boolean, boolean, boolean, String, String)

\*/

public static IScanner createScanner(boolean tokenizeComments, boolean tokenizeWhiteSpace, boolean assertMode, boolean recordLineSeparator)\{

// use default workspace compliance

long complianceLevelValue = CompilerOptions.versionToJdkLevel(JavaCore.getOption(JavaCore.COMPILER\_COMPLIANCE));

if (complianceLevelValue == 0) complianceLevelValue = ClassFileConstants.JDK1\_4; // fault-tolerance

PublicScanner scanner =

new PublicScanner(

tokenizeComments,

tokenizeWhiteSpace,

false/\*nls\*/,

assertMode ? ClassFileConstants.JDK1\_4 : ClassFileConstants.JDK1\_3/\*sourceLevel\*/,

complianceLevelValue,

null/\*taskTags\*/,

null/\*taskPriorities\*/,

true/\*taskCaseSensitive\*/);

scanner.recordLineSeparator = recordLineSeparator;

return scanner;

開發者ID:trylimits，項目名稱:Eclipse-Postfix-Code-Completion，代碼行數:53，

注：本文中的org.eclipse.jdt.internal.core.util.PublicScanner類示例整理自Github/MSDocs等源碼及文檔管理平台，相關代碼片段篩選自各路編程大神貢獻的開源項目，源碼版權歸原作者所有，傳播和使用請參考對應項目的License；未經允許，請勿轉載。

猜你喜欢

无需订阅费! 手把手教你如何在Amazon OpenSearch Service中使用跨集群复制功能（Cross-Cluster Replication）
手把手教你如何用Lambda + Alexa调用echo设备
EC2 Spot实例中断引起的AWS Batch任务重试优化
Python小技巧
Java-反射
新增 – 存储优化型 Amazon EC2 实例 (I4i)，由英特尔至强可扩展 (Ice Lake) 处理器提供支持
golang对比python
php对比python
java 集合
Python资源汇总
Amazon MSK Serverless 现已正式推出–无需再为托管式 Kafka 集群进行容量规划
多伦多的新 AWS Wavelength 区域 – 加拿大首个
AWS IoT TwinMaker 现已正式发布
python 列表操作
python 三级菜单
java基础
python 滚动字幕
java身份证验证代码IdcardUtils.java分享
Python---Tkinter---计算器
宣布 AWS Amplify Studio 正式发布

zl程序教程

当前栏目

java scanner getline,Java PublicScanner類代碼示例

相关文章